CN109002448B

CN109002448B - Report statistical method, device and system

Info

Publication number: CN109002448B
Application number: CN201710423452.4A
Authority: CN
Inventors: 姚海宏
Original assignee: China Mobile Communications Group Co Ltd; China Mobile Group Gansu Co Ltd
Current assignee: China Mobile Communications Group Co Ltd; China Mobile Group Gansu Co Ltd
Priority date: 2017-06-07
Filing date: 2017-06-07
Publication date: 2020-12-08
Anticipated expiration: 2037-06-07
Also published as: CN109002448A

Abstract

The invention discloses a report statistical method, a report statistical device and a report statistical system, which are used for solving the problems of high expansion cost, low statistical speed and poor performance of the conventional report system. The method comprises the following steps: the server receives a report form counting request which is sent by a terminal and carries a first field group; and matching the first field group with a second field group corresponding to each resource node, and when the existence of the target resource node is determined, sending the identification information of the target resource node and the SQL sentence generated according to the first field group to the terminal, so that the terminal sends the SQL sentence to the target resource node according to the identification information of the target resource node, and the target resource node sends a report statistical result to the terminal. The server does not carry out report statistics, so that the expansion cost of the server is saved, and the server distributes the report statistics request of each terminal to the corresponding target resource node without being intensively and uniformly completed by the server, so that the speed of report statistics at each time is increased.

Description

Report statistical method, device and system

Technical Field

The invention relates to the technical field of business support, in particular to a report statistical method, a device and a system.

Background

At present, a reporting System in the Business Support field is mainly provided by a Business Operation Support System (BOSS), a total architecture of the reporting System is mainly a C/S (client/server) architecture, when a client (terminal) needs to perform reporting statistics, a reporting statistics request is sent to a server, and after performing reporting statistics according to the reporting statistics request, the server sends a statistics result to the terminal. The data needed to be reported are stored in the server, and the counting process is also completed in the server.

With the development of the BOSS system, the business growth speed is faster and faster, new challenges are brought to the server, on one hand, the data volume needing to be stored is continuously increased, and pressure on the aspect of data storage is brought, and on the other hand, the statistical speed required by the client requesting report statistics is also faster and faster.

In order to meet the demand of service increase, the server may be vertically expanded and/or horizontally expanded according to different types of servers, where the vertical expansion may increase the storage capacity or the computing capacity of the original server, and the horizontal expansion may be a newly added server, for example. However, in either case, the cost required is very high. In addition, because the data needing to be counted is stored in a centralized mode, all data need to be traversed during counting, the single counting speed is hardly improved, even if the single counting speed is expanded, the concurrent counting speed can only be improved, and the effect on the single counting speed is not great.

In addition, the storage resource and the computing resource of the server are shared by all the terminals, the storage data volume is large, and under the condition of low computing speed, the resource of the server may be exhausted, so that other requirements of normal work of the server are influenced.

Disclosure of Invention

The embodiment of the invention discloses a report counting method, a report counting device and a report counting system, which are used for solving the problems of high expansion cost, low counting speed and poor performance of the conventional report system.

In order to achieve the above object, the embodiment of the present invention discloses a report statistics method, which comprises:

a server receives a report statistics request which is sent by a terminal and carries a first field group, wherein the first field group comprises at least one first field;

matching each first field in the first field group with each second field in a second field group corresponding to each pre-stored resource node, and determining whether a target resource node for performing report statistics exists;

if so, the identification information of the target resource node and the Structured Query Language (SQL) statement generated according to the first field group are sent to the terminal, so that the terminal sends the SQL statement to the target resource node according to the identification information of the target resource node, and the target resource node sends a report statistical result to the terminal.

Further, before matching each first field in the first field group with each second field in a second field group corresponding to each pre-stored resource node, the method further includes:

for each first field in the first field group, identifying a first category to which the first field belongs, and judging whether fragmentation information corresponding to the first category is stored; if not, carrying out the subsequent steps;

if yes, judging whether a third field successfully matched with the first field exists in the fragment information; and if a third field which is successfully matched exists, carrying out subsequent steps aiming at each resource node corresponding to the third field.

Further, if there is no third field for successful matching or there is no target resource node for report statistics, the method further includes:

and carrying out report statistics according to the SQL statement and first report data prestored by the terminal, and sending a report statistical result to the terminal.

Further, if each first field is successfully matched with each second field in a second field group corresponding to at least two pre-stored resource nodes, the determining a target resource node for performing report statistics includes:

and determining a target resource node for carrying out report statistics according to the current attribute information of the at least two resource nodes.

Further, the attribute information includes at least one of:

IP address, network speed, CPU utilization, and memory capacity.

Further, the method further comprises:

for each resource node, determining whether the resource node is an effective resource node according to the heartbeat frequency of the resource node; if not, deleting the stored identification information of the resource node and each second field in the corresponding second field group.

Further, the method further comprises:

receiving self attribute information sent by each terminal;

for each terminal, judging whether the terminal is used as a resource node or not according to the attribute information of the terminal; if so, sending the corresponding second report data to the terminal, and storing the corresponding relation between the identification information of the terminal and a second field group formed by each second field according to each second field corresponding to the sent second report data.

The embodiment of the invention discloses a report statistical system, which comprises: a first terminal, a server and a target resource node;

the first terminal is used for sending a report statistics request carrying a first field group to a server, wherein the first field group comprises at least one first field;

the server is used for receiving a report form counting request which is sent by a first terminal and carries a first field group; matching each first field in the first field group with each second field in a second field group corresponding to each pre-stored resource node, and determining whether a target resource node for performing report statistics exists; if so, sending the identification information of the target resource node and a Structured Query Language (SQL) statement generated according to the first field group to the first terminal;

the first terminal is further configured to send the SQL statement to the target resource node according to the identification information of the target resource node;

the target resource node is used for carrying out report statistics according to the SQL sentences sent by the first terminal and report data prestored by the target resource node, and sending a report statistical result to the first terminal;

and the first terminal is also used for receiving the report statistical result sent by the target resource node.

Further, the system further comprises: a second terminal;

the second terminal is used for sending the attribute information of the second terminal to the server;

the server is further configured to receive attribute information of the server sent by the second terminal, and determine whether to use the second terminal as a resource node according to the attribute information of the second terminal; if so, sending the corresponding report data to the second terminal, and storing the corresponding relation of the second field group formed by the identification information of the second terminal and each second field according to each second field corresponding to the report data;

and the second terminal is also used for receiving and storing the report data sent by the server.

The embodiment of the invention discloses a report statistics device, which comprises:

the system comprises a receiving module, a sending module and a receiving module, wherein the receiving module is used for a server to receive a report form counting request which is sent by a terminal and carries a first field group, and the first field group comprises at least one first field;

the matching determining module is used for matching each first field in the first field group with each second field in a second field group corresponding to each pre-stored resource node, and determining whether a target resource node for performing report statistics exists or not;

and the sending module is used for sending the identification information of the target resource node and the Structured Query Language (SQL) statement generated according to the first field group to the terminal when the matching result of the matching determining module is yes, so that the terminal sends the SQL statement to the target resource node according to the identification information of the target resource node, and the target resource node sends a report statistical result to the terminal.

Further, the apparatus further comprises:

a first determining module, configured to identify, for each first field in the first field group, a first category to which the first field belongs, and determine whether fragment information corresponding to the first category is stored; when the judgment result of the first judgment module is negative, triggering a matching determination module;

the second judging module is used for judging whether a third field successfully matched with the first field exists in the fragment information or not when the judging result of the first judging module is yes; and if the judgment result of the second judgment module is yes, triggering a matching determination module aiming at each resource node corresponding to the third field.

Further, the apparatus further comprises:

the statistical module is used for carrying out report statistics according to the SQL statement and the first report data prestored by the statistical module when the judgment result of the second judgment module is negative or the matching result of the matching determination module is negative;

and the sending module is also used for sending the report statistical result to the terminal.

Further, the matching determining module is specifically configured to determine, if each first field is successfully matched with each second field in a second field group corresponding to at least two pre-stored resource nodes, a target resource node for performing report statistics according to current attribute information of the at least two resource nodes.

Further, the apparatus further comprises:

the node management module is used for determining whether the resource node is an effective resource node or not according to the heartbeat frequency of the resource node aiming at each resource node; if not, deleting the stored identification information of the resource node and each second field in the corresponding second field group.

Further, the air conditioner is provided with a fan,

the receiving module is further configured to receive attribute information of each terminal sent by the terminal;

the device further comprises:

the distribution and storage module is used for judging whether the terminal is used as a resource node or not according to the attribute information of the terminal aiming at each terminal; if so, sending the corresponding second report data to the terminal, and storing the corresponding relation between the identification information of the terminal and a second field group formed by each second field according to each second field corresponding to the sent second report data.

The embodiment of the invention discloses a report statistical method, a device and a system, wherein the method comprises the following steps: a server receives a report statistics request which is sent by a terminal and carries a first field group, wherein the first field group comprises at least one first field; matching each first field in the first field group with each second field in a second field group corresponding to each pre-stored resource node, and determining whether a target resource node for performing report statistics exists; if so, the identification information of the target resource node and the Structured Query Language (SQL) statement generated according to the first field group are sent to the terminal, so that the terminal sends the SQL statement to the target resource node according to the identification information of the target resource node, and the target resource node sends a report statistical result to the terminal. In the embodiment of the invention, the server matches each first field in the report counting request with each second field corresponding to each pre-stored resource node, determines the target resource node, and provides the report counting result to the terminal through the target resource node. The server only generates SQL statements and determines target resource nodes, report counting is not carried out, so that the expansion cost of the server is saved, and the server distributes the report counting request of each terminal to the corresponding target resource nodes without being intensively and uniformly completed by the server, so that the speed of report counting at each time is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic diagram of a report statistics process provided in embodiment 1 of the present invention;

fig. 2 is a schematic diagram of a report statistics process provided in embodiment 3 of the present invention;

FIG. 3 is a diagram illustrating a report statistics process according to an embodiment of the present invention;

fig. 4 is a schematic diagram of report data for determining a resource node according to an embodiment of the present invention;

fig. 5 is a diagram of a report statistics system structure provided in embodiment 7 of the present invention;

FIG. 6 is a flow chart of a system according to an embodiment of the present invention;

fig. 7 is a structural diagram of a report statistics apparatus according to an embodiment of the present invention.

Detailed Description

In order to save the server expansion cost and improve the statistical speed and performance, the embodiment of the invention provides a report statistical method, a report statistical device and a report statistical system.

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Example 1:

fig. 1 is a schematic diagram of a report statistics process provided in embodiment 1 of the present invention, where the process includes the following steps:

s101: receiving a report statistics request which is sent by a terminal and carries a first field group, wherein the first field group comprises at least one first field.

The report form statistical method provided by the embodiment of the invention is applied to the server.

The method comprises the steps that a server receives a report counting request sent by a terminal, wherein the report counting request carries a first field group, and the first field group comprises at least one first field. The first field may be a statistical time, or a statistical time period, for example, 1/3/2017 or 1/2017-2/2017, a statistical item, for example, a birth population, or a statistical region, for example, Tianjin, Shanghai, etc.

S102: and matching each first field in the first field group with each second field in a second field group corresponding to each pre-stored resource node, determining whether a target resource node for performing report statistics exists, if so, performing S103, and if not, outputting prompt information.

Each second field in the second field group corresponding to each resource node is pre-stored in the server, specifically, the stored corresponding relationship between the identification information of each resource node and the second field group formed by each second field may be stored, and the stored corresponding relationship between the identification information of each resource node and the second field group formed by each second field may be stored in the resource node index table. And the second field group corresponding to each resource node is the combination of all fields corresponding to each report data stored in the resource node.

When a server receives a report statistics request carrying a first field group and sent by a terminal, each first field in the first field group can be matched with each second field in a second field group corresponding to each pre-stored resource node, and whether a target resource node for report statistics exists is determined. And if the server determines that the target resource node for performing report statistics does not exist, the server can output prompt information.

When a second field group corresponding to a certain resource node contains all first fields in the first field group, determining that each first field in the first field group is successfully matched with a pre-stored second field group corresponding to each resource node, and determining the resource node as a target resource node.

And the second field group corresponding to the resource node is related to the report data stored by the resource node. If the matching does not exist, the report data corresponding to the first field group is not stored in the resource node. If the matching exists, the corresponding target resource node is indicated to store the report data corresponding to the first field group, and report statistics can be performed.

S103: and sending the identification information of the target resource node and the SQL sentence generated according to the first field group to the terminal, so that the terminal sends the SQL sentence to the target resource node according to the identification information of the target resource node, and the target resource node sends a report statistical result to the terminal.

And if the server determines that the target resource node for performing report statistics exists, the server sends the identification information of the target resource node to the terminal. And the server can generate an SQL statement according to the first field group in the received report statistics request and send the SQL statement to the terminal. The process of generating the SQL statement by the server according to the first field group belongs to the prior art, and is not described in detail in the embodiment of the present invention.

And the server sends the identification information of the target resource node and the SQL sentence generated according to the first field group to the terminal, so that the terminal sends the SQL sentence to the target resource node according to the identification information of the target resource node, and the target resource node performs report statistics according to the SQL sentence and sends a report statistical result to the terminal.

When the target resource node is determined, if only the second field group corresponding to one resource node contains all the first fields in the first field group, the one resource node is directly determined as the target resource node.

And aiming at each resource node, multiple report data may be stored in the resource node, and the resource node corresponds to multiple second field groups, and when matching is performed, the first field group may be successfully matched with only one second field group, or may be successfully matched with multiple second field groups. If the matching with the plurality of second field groups is successful, the resource node can count the report data according to a plurality of reports.

In the embodiment of the invention, the server matches each first field in the report counting request with each second field corresponding to each pre-stored resource node, determines the target resource node, and provides the report counting result to the terminal through the target resource node. And the server only generates SQL statements and determines target resource nodes without performing report statistics, so that the extended cost of the server is saved, and the server allocates the report statistics request of each terminal to the corresponding target resource nodes without being intensively and uniformly completed by the server, so that the speed of report statistics at each time is improved.

Example 2:

in order to increase the speed of determining whether there is a target resource node for performing report statistics, and thus further increase the speed of report statistics, on the basis of the foregoing embodiment, in an embodiment of the present invention, before matching each first field in the first field group with each second field in a second field group corresponding to each pre-stored resource node, the method further includes:

Because the report data stored in the server generally contains fields such as time, item, area and the like, the report data can be fragmented according to the category to which the fields belong, so that query and statistics are facilitated. For example, when the type of the field is a region, all report data corresponding to the field containing the region type may be fragmented, all report data in the fragment contains information about the region, and the corresponding field is saved for each fragment according to the difference of the regions contained.

Specifically, in the embodiment of the present invention, the server may record which fields of which categories store the fragmentation information, and record which third fields the corresponding fragments all contain in the fragmentation information. In addition, for each third field, the server also stores each resource node corresponding to the third field and each second field in the second field group corresponding to each resource node. A third field corresponding to each slice may be stored in the slice index table.

If the server is fragmented according to the type of the region, the third field corresponding to the fragmentation information can be the regions such as Tianjin and Beijing, and the server performs fragmentation processing on the stored report data of the regions such as Tianjin and Beijing, and takes each region as the third field. When the third field is Tianjin, the server stores report data containing the field "Tianjin", identification information of resource nodes for counting the report data containing the field "Tianjin", and a second field group corresponding to each report data, namely each resource node corresponding to Tianjin and a second field group corresponding to each resource node are stored; when the third field is beijing, the server stores report data including the field "beijing", identification information of resource nodes for counting the report data including the field "beijing", and a second field group corresponding to each report data, that is, each resource node corresponding to beijing and a second field group corresponding to each resource node are stored.

After receiving a report statistics request carrying a first field group and sent by a terminal, a server can determine the category to which each first field in the first field group belongs, perform fragmentation information screening according to the category to which each first field belongs and the category corresponding to each piece of stored fragmentation information, and then determine a target resource node in the screened fragmentation information. Specifically, for each first field in the first field group, a first category to which the first field belongs may be identified, and whether fragmentation information corresponding to the first category is stored is determined; if the fragment information corresponding to the first class is stored, judging whether a third field successfully matched with the first field exists in the fragment information; and if a third field successfully matched with the first field exists, matching each first field in the first field group with each second field in a second field group corresponding to each pre-stored resource node corresponding to the third field aiming at each resource node corresponding to the third field, and determining whether a target resource node for performing report statistics exists.

When the server determines that a third field successfully matched with the first field exists, the fragment corresponding to the third field can be called a target fragment.

When determining the third field corresponding to the fragmentation information, the server may preset a category corresponding to the fragmentation information, and determine a field in the second field group in the fragmentation information, which conforms to the category, as the third field corresponding to the fragmentation information.

If the server judges that the fragmentation information corresponding to the first class corresponding to any one of the first fields is not stored, it indicates that fragmentation is not performed for the first class corresponding to any one of the first fields, and may directly match each first field in the first field group with each second field in a second field group corresponding to each pre-stored resource node, to determine whether a target resource node for performing report statistics exists.

If the fragmentation information corresponding to the first class is stored, but a third field successfully matched with the first field corresponding to the first class does not exist in the fragmentation information, it indicates that fragmentation is performed according to the first class, but a third field successfully matched with the first field does not exist in the fragmentation information corresponding to the first class, and thus a target resource node does not exist.

Example 3:

when the server determines that the target resource node does not exist, the server can perform report statistics on report data stored by the server in order to provide a corresponding statistical result for the user, and returns the statistical result to the terminal. On the basis of the foregoing embodiments, in an embodiment of the present invention, if there is no third field that is successfully matched or there is no target resource node for performing report statistics, the method further includes:

and performing report statistics according to the SQL sentences generated by the first field group and first report data prestored by the terminal, and sending the report statistics result to the terminal.

In the embodiment of the invention, if the server determines that the third field successfully matched with the first field does not exist or the target resource node for performing report statistics does not exist, the server indicates that the server does not store the report data corresponding to the first field into the resource node, and the server performs report statistics according to the SQL statement generated by the first field group and the first report data prestored by the server, and sends the report statistics result to the terminal.

When determining that the target resource node for performing report statistics does not exist, the server can determine that the target resource node for performing report statistics is matched with each first field in the first field group directly and each second field in the second field group corresponding to each pre-stored resource node;

or after judging that the fragmentation information corresponding to the first class is not stored, matching each first field in the first field group with each second field in a second field group corresponding to each pre-stored resource node to determine;

the fragmentation information corresponding to the first field may be determined to be stored, and after a third field successfully matched with the first field exists in the fragmentation information, for each resource node corresponding to the third field, matching is performed between each first field in the first field group and each second field in the second field group corresponding to each resource node corresponding to the third field in advance.

Fig. 2 is a schematic diagram of a report statistics process provided in embodiment 3 of the present invention, where the process includes the following steps:

s201: receiving a report statistics request which is sent by a terminal and carries a first field group, wherein the first field group comprises at least one first field.

S202: and for each first field in the first field group, identifying a first category to which the first field belongs, judging whether fragment information corresponding to the first category is stored, if so, executing S203, and if not, executing S204.

S203: and judging whether a third field successfully matched with the first field exists in the fragment information, if so, executing S204 aiming at each resource node corresponding to the third field, and if not, executing S206.

S204: and matching each first field in the first field group with each second field in a second field group corresponding to each pre-stored resource node, determining whether a target resource node for performing report statistics exists, if so, executing S205, and if not, executing S206.

S205: and sending the identification information of the target resource node and the SQL sentence generated according to the first field group to the terminal, so that the terminal sends the SQL sentence to the target resource node according to the identification information of the target resource node, and the target resource node sends a report statistical result to the terminal.

S206: and performing report statistics according to the SQL sentences generated by the first field group and first report data prestored by the terminal, and sending the report statistics result to the terminal.

Example 4:

if a second field group corresponding to at least two resource nodes contains all first fields in the first field group, any one of the at least two resource nodes can be determined as a target resource node. In order to make the performance of the determined target resource node more excellent, the statistical performance is improved. On the basis of the foregoing embodiments, in an embodiment of the present invention, if each first field is successfully matched with each second field in a second field group corresponding to at least two pre-stored resource nodes, the determining a target resource node for performing report statistics includes:

The attribute information includes at least one of: IP address, network speed, CPU utilization, and memory capacity.

When the attribute information is the IP address, the server may determine, according to the IP address of the terminal and the IP addresses of the at least two resource nodes, that the resource node closest to the terminal is a target resource node; or the like, or, alternatively,

when the attribute information is the network speed, the server detects the network speeds of the at least two resource nodes and determines the resource node with the highest network speed as a target resource node; or the like, or, alternatively,

when the attribute information is the CPU utilization rate and the memory capacity, the server detects the CPU utilization rate and the memory capacity of the at least two resource nodes, and determines a target resource node in the at least two resource nodes according to a preset weighting algorithm; or

And the server determines a target resource node in the at least two resource nodes according to the IP address of the terminal, the IP addresses of the at least two resource nodes, the network speed of the at least two resource nodes, the CPU utilization rate and the memory capacity of the at least two resource nodes and a preset weighting algorithm.

As shown in the report statistics process diagram shown in fig. 3, the server receives a report statistics request carrying a first field group sent by a terminal, generates an SQL statement according to the first field group in the report statistics request, and determines whether a target fragment exists according to the fragment index table. And if the target fragment does not exist, performing report statistics according to the report data of the terminal and the generated SQL statement, and sending a report statistical result to the terminal.

And if the target fragment exists, determining whether a target resource node exists according to the resource node index table. And if the target resource node does not exist, performing report statistics according to the report data of the terminal and the generated SQL statement, and sending a report statistical result to the terminal. And if one resource node capable of performing report statistics exists, directly taking the resource node as a target resource node, and sending the identification information of the target resource node and the generated SQL statement to the terminal. And if at least two resource nodes capable of carrying out report statistics exist, selecting one resource node as a target resource node according to the attribute information of the resource nodes, and sending the identification information of the target resource node and the generated SQL statement to the terminal.

Example 5:

after receiving the report statistics request, the server determines a target resource node corresponding to the report statistics request, before that, the server needs to determine a corresponding terminal as a resource node, determine report data stored in each resource node, and extract field information according to the data stored in each resource node for storage, so as to perform the process of determining the target resource node corresponding to the statistics request. On the basis of the foregoing embodiments, in an embodiment of the present invention, the method further includes:

receiving self attribute information sent by each terminal;

for each terminal, judging whether the terminal is used as a resource node or not according to the attribute information of the terminal; if so, sending the corresponding second report data to the terminal, and storing the corresponding relation between the identification information of the terminal and a second field group formed by each second field according to each second field corresponding to the second report data.

In the embodiment of the present invention, when determining the resource node, the server may determine the resource node in a dynamic manner, that is, determine whether to determine the terminal as the resource node according to the attribute information of each terminal, specifically, the server may receive the attribute information of itself sent by each terminal; and for each terminal, judging whether the terminal is used as a resource node or not according to the attribute information of the terminal. The attribute information includes at least one of: IP address, network speed, CPU utilization, and memory capacity.

The terminal sends the attribute information of itself to the server, which may be sent when sending the report statistics request, or sent according to a preset period.

When the server determines that the terminal is a resource node, the server may send corresponding second report data to the terminal, and store a corresponding relationship between the identification information of the terminal and a second field group formed by each second field according to each second field corresponding to the second report data.

When determining the second report data corresponding to the resource node, the server may determine according to the remaining storage capacity of the resource node and the storage capacity occupied by the report data, where the storage capacity occupied by each report data may be different, may sort the storage capacity occupied by each report data first, and then distribute the report data, where a specific determination process is not limited herein.

When the server determines the resource node, the server may also determine the resource node in a static manner, that is, the server stores the identification information of each resource node according to the configuration of the user. Generally, in the report statistics service, statistics has periodicity, and in one period, report data is almost fixed, and the storage capacity occupied by the report data is also fixed, so that a fixed number of resource nodes can be selected from various cities according to the storage capacity occupied by the report data, and the corresponding relation between the identification information of the resource node and a second field group formed by each second field is stored according to second report data stored by each resource node.

When determining the resource nodes, the server may also determine in a manner of combining static and dynamic states, and determine a first number and a second number according to the total number of the resource nodes required, where the sum of the first number and the second number is the total number, and the first number and the second number may be the same or different. According to the storage capacity occupied by all the report data, a user selects a first number of resource nodes from various cities and configures the resource nodes into a server to serve as the resource nodes, but the first number of resource nodes cannot store all the report data in the server, and the server can determine a second number of resource nodes according to the attribute information of the server sent by the terminal so as to ensure that all the report data can be stored in the corresponding resource nodes.

When the report data corresponding to the resource node is determined, the server can directly determine according to the residual storage capacity of the resource node and the size of the storage capacity occupied by each report data. If the server fragments the report data, determining which resource nodes store the report data in the fragment according to the storage capacity occupied by the report data in each fragment and the residual storage capacity of each resource node, and then distributing the report data in the fragment to the corresponding resource nodes.

For example, if the remaining storage capacity of each resource node is a, when all report data is fragmented, the data is divided into D fragments, the number B of reports in each fragment is the same, and the storage capacity C occupied by each report is the same, then: and the number X of the resource nodes is equal to the storage capacity C occupied by a single report, the fragment number D, the number B of each fragment report and the residual storage capacity A of the resource nodes.

When a certain resource node storing report data in a certain fragment has a problem, in order to ensure normal operation of report statistics, multiple copies of the report data in the fragment can be stored, that is, the same report data is distributed to at least two resource nodes for storage, and when one resource node has a problem, the other resource nodes can perform report statistics.

If the number of copies of each report stored in the fragment is referred to as redundancy Y, the total number of resource nodes required is X × Y.

Before the report data is subjected to the fragment processing, the report data can be sorted according to the report statistical aperture to generate intermediate data, for example, the report data is adjusted according to time or region. And generating intermediate data and then carrying out fragmentation processing, so that the target resource node is ensured to count in single report data as much as possible when carrying out report data statistics, and the complexity of target resource node statistics is reduced.

As shown in fig. 4, the schematic diagram of determining the report data of the resource node is that all the report data are divided into 1-N segments by generating intermediate data according to the statistical aperture of the report and performing segmentation processing on the intermediate data according to the regionality. And aiming at the fragment 1, distributing the report data in the fragment 1 to 1-M resource nodes. The statistical caliber of the report is a certain field in the report data. Similarly, the above process also exists for other fragments, and the report data in each fragment is distributed to the corresponding resource node, where the resource nodes corresponding to each fragment may be different, or may be partially or completely the same.

Example 6:

after determining the resource node, the server may further manage the resource node, and based on the foregoing embodiments, in an embodiment of the present invention, the method further includes:

In the embodiment of the invention, a resource node sends a heartbeat detection packet to a server, for each resource node, the server stores the frequency of receiving the heartbeat detection packet sent by the resource node, namely the heartbeat frequency of the resource node, if the server detects that the frequency of the received heartbeat detection packet does not accord with the stored frequency, the resource node is determined not to be an effective resource node, the resource node is invalid, report statistics cannot be carried out, and the server deletes the stored identification information of the invalid resource node and each second field in a second field group corresponding to the invalid resource node.

And simultaneously, a report data clearing instruction can be sent to the failed resource node, so that the failed resource node clears the report data stored by the failed resource node, and the requirements of other works are met.

Example 7:

fig. 5 is a structural diagram of a report statistics system provided in embodiment 7 of the present invention, where the system includes: a first terminal 51, a server 52 and a target resource node 53;

the first terminal 51 is configured to send a report statistics request carrying a first field group to the server 52, where the first field group includes at least one first field;

the server 52 is configured to receive a report statistics request carrying a first field group sent by the first terminal 51; matching each first field in the first field group with each second field in a second field group corresponding to each pre-stored resource node, and determining whether a target resource node 53 for performing report statistics exists; if yes, the identification information of the target resource node 53 and the Structured Query Language (SQL) statement generated according to the first field group are sent to the first terminal 51;

the first terminal 51 is further configured to send the SQL statement to the target resource node 53 according to the identification information of the target resource node 53;

the target resource node 53 is configured to perform report statistics according to the SQL statements sent by the first terminal 51 and report data prestored by the target resource node, and send a report statistical result to the first terminal 51;

the first terminal 51 is further configured to receive a report statistic result sent by the target resource node 53.

The report statistics system provided by the embodiment of the invention comprises a first terminal, a server and a target resource node.

When the first terminal identifies that the report counting request exists, the report counting request carrying a first field group is sent to the server, wherein the first field group comprises at least one first field. Wherein the first field set includes at least one first field. The first field may be a statistical time, or a statistical time period, for example, 1/3/2017 or 1/2017-2/2017, a statistical item, for example, a birth population, or a statistical region, for example, Tianjin, Shanghai, etc.

When the server receives a report statistics request carrying a first field group and sent by a first terminal, each first field in the first field group can be matched with each second field in a second field group corresponding to each pre-stored resource node, and whether a target resource node for report statistics exists is determined. And if the server determines that the target resource node for performing report statistics does not exist, the server can output prompt information.

And if the server determines that the target resource node for performing report statistics exists, sending the identification information of the target resource node to the first terminal. And the server can generate an SQL statement according to the first field group in the received report statistics request and send the SQL statement to the first terminal. The process of generating the SQL statement by the server according to the first field group belongs to the prior art, and is not described in detail in the embodiment of the present invention.

The server sends the identification information of the target resource node and the SQL sentence generated according to the first field group to the first terminal, the first terminal sends the SQL sentence to the target resource node according to the identification information of the target resource node, the target resource node carries out report statistics according to the SQL sentence sent by the first terminal and report data prestored by the target resource node, report statistics results are sent to the first terminal, and the first terminal receives the report statistics results sent by the target resource node and displays the report statistics results.

When the server determines a target resource node, if only a second field group corresponding to one resource node contains all first fields in the first field group, the server directly determines the resource node as the target resource node. If a second field group corresponding to at least two resource nodes contains all first fields in the first field group, any one of the at least two resource nodes can be determined as a target resource node.

In the embodiment of the invention, the server matches each first field in the report counting request with each second field corresponding to each pre-stored resource node, determines the target resource node, and sends the report counting result to the terminal through the target resource node. The server only generates SQL sentences and determines target resource nodes without performing report statistics, so that the extended cost of the server is saved, and report statistics requests of each terminal are distributed to the corresponding target resource nodes by the server without being intensively and uniformly completed by the server, so that the speed of report statistics at each time is increased.

Example 8:

after receiving the report statistics request, the server determines a target resource node corresponding to the report statistics request, before that, the server needs to determine a corresponding terminal as a resource node, determine report data stored in each resource node, and extract field information according to the data stored in each resource node for storage, so as to perform the process of determining the target resource node corresponding to the statistics request. On the basis of the above embodiment 7, in an embodiment of the present invention, the system further includes: the second terminal is used for sending the attribute information of the second terminal to the server;

In the embodiment of the present invention, when determining the resource node, the server may determine the resource node in a dynamic manner, that is, determine whether to determine the second terminal as the resource node according to the attribute information of each second terminal, specifically, the second terminal may send its own attribute information to the server, and the server receives its own attribute information sent by each second terminal; and for each second terminal, judging whether the second terminal is used as a resource node or not according to the attribute information of the second terminal. The attribute information includes at least one of: IP address, network speed, CPU utilization, and memory capacity.

The second terminal sends the attribute information of itself to the server, which may be sent when the report statistics request is sent, or sent according to a preset period.

When the server determines that the second terminal is a resource node, the server may send corresponding second report data to the second terminal, and store a corresponding relationship between the identification information of the second terminal and a second field group formed by each second field according to each second field corresponding to the second report data.

When the server determines the resource node, the server may also determine the resource node in a static determination manner, that is, the server stores the identification information of each resource node according to the configuration of the user. Generally, in the report statistics service, statistics has periodicity, and in one period, report data is almost fixed, and the storage capacity occupied by the report data is also fixed, so that a fixed number of resource nodes can be selected from various cities according to the storage capacity occupied by the report data, and the corresponding relation between the identification information of the resource node and a second field group formed by each second field is stored according to second report data stored by each resource node.

When determining the resource nodes, the server may also determine in a manner of combining static and dynamic states, and determine a first number and a second number according to the total number of the resource nodes required, where the sum of the first number and the second number is the total number, and the first number and the second number may be the same or different. According to the storage capacity occupied by all the report data, the user selects a first number of resource nodes from various cities and configures the resource nodes into the server as the resource nodes, but the first number of resource nodes cannot store all the report data in the server, and the server can determine a second number of resource nodes according to the own attribute information sent by the second terminal so as to ensure that all the report data can be stored into the corresponding resource nodes.

The first and second terminals mentioned above may be the same or different.

Example 9:

after the server determines the resource node, the server may also manage the resource node, and on the basis of embodiments 7 and 8, in this embodiment of the present invention, if the terminal is the resource node, the terminal is further configured to send a heartbeat detection packet to the server;

the server is further configured to receive a heartbeat detection packet sent by the terminal, and determine whether the terminal is an effective resource node according to the frequency of the received heartbeat detection packet and a preset target frequency; if not, deleting each second field in the stored identification information of the terminal and the corresponding second field group.

In the embodiment of the invention, a resource node sends a heartbeat detection packet to a server, for each resource node, the server stores the frequency of receiving the heartbeat detection packet sent by the resource node, if the server detects that the frequency of the received heartbeat packet does not accord with the stored frequency, the resource node is determined not to be an effective resource node, the resource node is invalid, report statistics cannot be carried out, and the server deletes the stored identification information of the invalid resource node and each second field in a second field group corresponding to the invalid resource node.

The server can also send a report data clearing instruction to the failed resource node, and the failed resource node clears the self-stored report data after receiving the clear instruction so as to meet the requirements of other works.

The terminal sending the report statistics request may be a resource node or not, and when the terminal is a resource node, the terminal may be a target resource node or not.

Fig. 6 is a system flowchart provided in the embodiment of the present invention, in which a server determines a resource node, manages the resource node, and sends report data to the resource node. When the terminal sends a report statistics request to the server, the server returns the identification information of the target resource node and the SQL sentence, the terminal sends the SQL sentence to the target resource node according to the identification information of the resource node, and the target resource node performs report statistics according to the report data stored by the target resource node and the received SQL sentence and sends a report statistics result to the terminal.

According to the above description of the embodiments, it can be summarized that the server has the functions of: the management of the resource node comprises the following steps: heartbeat detection and performance evaluation; distribution of data, comprising: carrying out fragment processing and distribution processing; determining a target resource node, generating an SQL statement and the like.

If the terminal is a common terminal, the functions are as follows: sending report statistics request, positioning target resource node, sending attribute information and the like.

If the terminal is a resource node, on the basis of a common terminal, the terminal also has the functions of report statistics and sending heartbeat detection packets.

Fig. 7 is a structural diagram of a report statistics apparatus according to an embodiment of the present invention, where the apparatus includes:

a receiving module 71, configured to receive, by a server, a report statistics request that is sent by a terminal and carries a first field group, where the first field group includes at least one first field;

a matching determining module 72, configured to match each first field in the first field group with each second field in a second field group corresponding to each pre-stored resource node, and determine whether a target resource node for performing report statistics exists;

and the sending module 73 is configured to send, when the matching result of the matching determining module is yes, the identification information of the target resource node and a Structured Query Language (SQL) statement generated according to the first field group to the terminal, so that the terminal sends the SQL statement to the target resource node according to the identification information of the target resource node, and the target resource node sends a report statistic result to the terminal.

The device further comprises:

a first determining module 74, configured to identify, for each first field in the first field group, a first category to which the first field belongs, and determine whether fragment information corresponding to the first category is stored; when the judgment result of the first judgment module 74 is negative, triggering the matching determination module 72;

a second judging module 75, configured to, when the judgment result of the first judging module 74 is yes, judge whether a third field successfully matched with the first field exists in the fragmentation information; if the judgment result of the second judgment module 75 is yes, the matching determination module 72 is triggered for each resource node corresponding to the third field.

The device further comprises:

a statistical module 76, configured to perform report statistics according to the SQL statement and the first report data pre-stored by the statistical module 76 when the determination result of the second determining module 75 is negative or the matching result of the matching determining module 72 is negative;

the sending module 73 is further configured to send the report statistic result to the terminal.

The matching determining module 72 is specifically configured to determine a target resource node for performing report statistics according to the current attribute information of the at least two resource nodes if each first field is successfully matched with each second field in the second field group corresponding to the at least two pre-stored resource nodes.

The device further comprises:

a node management module 77, configured to determine, for each resource node, whether the resource node is an effective resource node according to the heartbeat frequency of the resource node; if not, deleting the stored identification information of the resource node and each second field in the corresponding second field group.

The receiving module 71 is further configured to receive attribute information of each terminal sent by the terminal;

the device further comprises:

a distribution saving module 78, configured to determine, for each terminal, whether to use the terminal as a resource node according to the attribute information of the terminal; if so, sending the corresponding second report data to the terminal, and storing the corresponding relation between the identification information of the terminal and a second field group formed by each second field according to each second field corresponding to the sent second report data.

For the system/apparatus embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference may be made to some descriptions of the method embodiments for relevant points.

It is to be noted that, in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or operation from another entity or operation without necessarily requiring or implying any actual such relationship or order between such entities or operations.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely application embodiment, or an embodiment combining application and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. A report statistical method is characterized in that the method comprises the following steps:

if so, sending the identification information of the target resource node and a Structured Query Language (SQL) statement generated according to the first field group to the terminal, so that the terminal sends the SQL statement to the target resource node according to the identification information of the target resource node, and the target resource node sends a report statistical result to the terminal;

wherein, if there is no target resource node for performing report statistics, the method further comprises:

performing report statistics according to the SQL statement and first report data prestored by the terminal, and sending a report statistical result to the terminal;

wherein, if each first field is successfully matched with each second field in a second field group corresponding to at least two pre-stored resource nodes, the determining a target resource node for performing report statistics includes:

determining a target resource node for carrying out report statistics according to the current attribute information of the at least two resource nodes; the attribute information includes at least one of: IP address, network speed, CPU utilization, and memory capacity.

2. The method of claim 1, wherein prior to matching each first field of the first set of fields with each second field of a second set of fields corresponding to each pre-stored resource node, the method further comprises:

3. The method of claim 2, wherein if there is no third field for which the matching is successful, the method further comprises:

4. The method of claim 1, wherein the method further comprises:

5. The method of claim 1 or 4, wherein the method further comprises:

receiving self attribute information sent by each terminal;

6. A report statistics system, the system comprising: a first terminal, a server and a target resource node;

the first terminal is also used for receiving a report statistical result sent by the target resource node;

the server is also used for carrying out report statistics according to the SQL statement and first report data prestored by the server when a target resource node for carrying out report statistics does not exist, and sending a report statistical result to the terminal;

wherein, if each first field is successfully matched with each second field in a second field group corresponding to at least two pre-stored resource nodes, the server is specifically configured to: determining a target resource node for carrying out report statistics according to the current attribute information of the at least two resource nodes; the attribute information includes at least one of: IP address, network speed, CPU utilization, and memory capacity.

7. The system of claim 6, wherein the system further comprises: a second terminal; the second terminal is used for sending the attribute information of the second terminal to the server;

8. A report statistics apparatus, the apparatus comprising:

the report statistics system comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving a report statistics request which is sent by a terminal and carries a first field group, and the first field group comprises at least one first field;

a sending module, configured to send, when a matching result of the matching determining module is yes, the identification information of the target resource node and a Structured Query Language (SQL) statement generated according to the first field group to the terminal, so that the terminal sends the SQL statement to the target resource node according to the identification information of the target resource node, and the target resource node sends a report statistics result to the terminal;

the statistical module is used for carrying out report statistics according to the SQL statement and first report data prestored by the statistical module when the matching result of the matching determining module is negative;

the sending module is also used for sending the report statistical result to the terminal;

the matching determining module is specifically configured to determine a target resource node for performing report statistics according to current attribute information of at least two resource nodes if each first field is successfully matched with each second field in a second field group corresponding to at least two pre-stored resource nodes; the attribute information includes at least one of: IP address, network speed, CPU utilization, and memory capacity.

9. The apparatus of claim 8, wherein the apparatus further comprises:

10. The apparatus according to claim 9, wherein the statistical module is further configured to perform report statistics according to the SQL statement and the first report data pre-stored by the statistical module when the determination result of the second determining module is negative;

11. The apparatus of claim 8, wherein the apparatus further comprises:

12. The apparatus of claim 8 or 11,

the device further comprises: