CN113377604A

CN113377604A - Data processing method, device, equipment and storage medium

Info

Publication number: CN113377604A
Application number: CN202010163308.3A
Authority: CN
Inventors: 李洋; 张希明
Original assignee: Beijing Jingdong Zhenshi Information Technology Co Ltd
Current assignee: Beijing Jingdong Zhenshi Information Technology Co Ltd
Priority date: 2020-03-10
Filing date: 2020-03-10
Publication date: 2021-09-10
Anticipated expiration: 2040-03-10
Also published as: CN113377604B

Abstract

The embodiment of the invention discloses a data processing method, a device, equipment and a storage medium, wherein the method comprises the following steps: receiving index configuration information corresponding to a service index sent by a client, wherein the index configuration information is obtained by a user through configuration based on the service index caliber in advance; inputting the index configuration information into a preset data statistical code; and obtaining a statistical result corresponding to the service index by executing the input data statistical code. Through the technical scheme of the embodiment of the invention, developers can be prevented from repeatedly writing index statistical codes, so that the labor cost is saved, and the development efficiency is improved.

Description

Data processing method, device, equipment and storage medium

Technical Field

Embodiments of the present invention relate to computer technologies, and in particular, to a data processing method, an apparatus, a device, and a storage medium.

Background

With the increasing business demands, it is often necessary to analyze and count business data by using a plurality of business index apertures, so that business personnel can better understand the business development situation, wherein the business index apertures are a statistical rule for the business data.

In the prior art, when a new index caliber is added, a developer needs to compile a corresponding statistical logic code based on the new index caliber, and test the statistical logic code on line, so that data statistics can be performed by using the statistical logic code on line to obtain a statistical result of the index.

However, in the process of implementing the present invention, the inventor finds that at least the following problems exist in the prior art:

due to the fact that the statistical logic corresponding to the index aperture is complex and the statistical logic is solidified in a code form in the prior art, developers need to rewrite new statistical logic codes every time a new index aperture is added, time and labor are wasted, and development efficiency is reduced.

Disclosure of Invention

The embodiment of the invention provides a data processing method, a data processing device, data processing equipment and a storage medium, so that developers are prevented from repeatedly compiling index statistical codes, the labor cost is saved, and the development efficiency is improved.

In a first aspect, an embodiment of the present invention provides a data processing method, including:

receiving index configuration information corresponding to a service index sent by a client, wherein the index configuration information is obtained by a user through configuration based on the caliber of the service index in advance;

inputting the index configuration information into a preset data statistics code;

and obtaining a statistical result corresponding to the service index by executing the input data statistical code.

In a second aspect, an embodiment of the present invention further provides a data processing apparatus, including:

the system comprises an index configuration information receiving module, a service index configuration information sending module and a service index configuration information receiving module, wherein the index configuration information is obtained by a user through configuration based on the service index caliber in advance;

the index configuration information input module is used for inputting the index configuration information into a preset data statistical code;

and the statistical result obtaining module is used for obtaining a statistical result corresponding to the service index by executing the input data statistical code.

In a third aspect, an embodiment of the present invention further provides an apparatus, where the apparatus includes:

one or more processors;

a memory for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement a data processing method as provided by any of the embodiments of the invention.

In a fourth aspect, the embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the data processing method provided in any embodiment of the present invention.

The embodiment of the invention has the following advantages or beneficial effects:

the user can configure the service index in the client in advance based on the service index aperture to obtain corresponding index configuration information, so that the client sends the index configuration information to the application server. The application server is preset with configurable data statistical codes, so that when the index configuration information sent by the client is obtained, the index configuration information can be input into the data statistical codes, the dynamic configuration of the codes is realized, and then the statistical results corresponding to the service indexes can be obtained by executing the configured data statistical codes, so that developers do not need to repeatedly write the codes and do not need to re-deploy the codes on line, and only need to configure the codes in the client to obtain the corresponding index configuration information, thereby saving the labor cost and improving the development efficiency.

Drawings

Fig. 1 is a flowchart of a data processing method according to an embodiment of the present invention;

fig. 2 is a flowchart of a current statistical result determining process in a data processing method according to a second embodiment of the present invention;

FIG. 3 is an example of stored key-value pair information according to a second embodiment of the present invention;

FIG. 4 is an example of another stored key-value pair information according to a second embodiment of the present invention;

fig. 5 is a flowchart of a current statistical result determining process in a data processing method according to a third embodiment of the present invention;

fig. 6 is an example of scheduling parameter configuration information corresponding to a data update task according to a third embodiment of the present invention;

fig. 7 is a flowchart of a current statistical result determining process in a data processing method according to a fourth embodiment of the present invention;

fig. 8 is a schematic structural diagram of a data processing apparatus according to a fifth embodiment of the present invention;

fig. 9 is a schematic structural diagram of an apparatus according to a sixth embodiment of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.

Example one

Fig. 1 is a flowchart of a data processing method according to an embodiment of the present invention, which is applicable to a situation of performing data statistics on a service indicator. The method may be performed by a data processing apparatus, which may be implemented in software and/or hardware, integrated in an application server having means for managing business data. As shown in fig. 1, the method specifically includes the following steps:

s110, receiving index configuration information corresponding to the service index sent by the client, wherein the index configuration information is obtained by a user through configuration based on the service index caliber in advance.

The service index caliber may refer to a statistical rule set by a service worker based on service requirements and used for statistical service data. The service index can refer to an index identifier which needs to be processed in the service index caliber in a statistical manner. For example, the business index calibers may be: the rules for counting sales during the promotion event include, for example, counting orders whose order status is valid, and the location of the user is north China, and counting at a counting interval of every day. The corresponding business indicator may be "sales". The index configuration information may be configuration information obtained by configuring a service index based on a service index aperture in the client.

Specifically, a configuration page for a user to perform index configuration is preset in the client. The configuration page can contain each configuration item corresponding to the service index, so that a user can obtain the configuration information corresponding to each configuration item based on the service index aperture, input the corresponding configuration information in each configuration item in the configuration page, and after the input is finished, the configured index configuration information can be sent to the application server in a button clicking mode, so that the application server can receive the index configuration information sent by the client. The configuration page can also support the uploading function of the index configuration file, so that a user can write a corresponding index configuration file based on each configuration information obtained by the service index aperture, and the configuration operation of the service index can be rapidly completed by uploading the index configuration file on the configuration page. When a plurality of service indexes need to be configured, the batch configuration operation of the plurality of service indexes can be completed in a mode of uploading corresponding index configuration files in batches, so that the configuration efficiency is improved. The indicator configuration file can be described in a format such as but not limited to JSON (JavaScript Object Notation).

It should be noted that each configuration item in the index configuration information may be set in advance based on a service requirement, so as to meet different statistical requirements of users. Illustratively, the metric configuration information may include, but is not limited to: and counting time information, data filtering conditions and data counting modes. The statistical time information can be used to characterize the user's requirement for statistical time, such as statistical daily or hourly data. The data filtering condition may refer to a condition satisfied by data to be filtered out in the service data, or a condition satisfied by specific service data to be counted, so as to determine target service data to be counted currently. The data statistics manner may refer to a manner of performing statistics on the target service data, for example, the data statistics manner may be, but is not limited to, a sum manner or a count manner. It should be noted that whether the configuration of the statistical time information and the data filtering condition is optional, and for some service scenarios that do not need to consider a time boundary, the statistical time information and the data filtering condition do not need to be configured in a configuration page, and at this time, the blanking process may be performed.

And S120, inputting the index configuration information into a preset data statistics code.

The preset data statistics code can be uniformly preset in advance based on each configuration item in the configuration page, and does not contain a data statistics logic framework of specific configuration information, so as to implement the configurability of the code.

Specifically, after receiving the index configuration information sent by the client, the application server can input the index configuration information into the preset data statistics code so as to obtain a complete executable data statistics code, and realize dynamic configuration of the code, so that developers do not need to repeatedly write the statistics code, the labor cost is saved, and the development efficiency is improved.

And S130, acquiring a statistical result corresponding to the service index by executing the input data statistical code.

The statistical result corresponding to the service index may be data obtained by performing statistics on the target service data. For example, if the service index is: and selling amount, wherein the statistical result corresponding to the service index is the specific selling amount.

Specifically, the application server executes the data statistics code after the index configuration information is input, and can perform corresponding data statistics logic based on the index configuration information, so as to quickly obtain a statistics result corresponding to the service index. In the embodiment, the user only needs to complete the configuration operation of the service index based on the service index aperture in the client, and the application server can automatically determine the statistical result corresponding to the service index based on the index configuration information, so that the user operation is simplified, the online index statistics is realized, and the development efficiency is improved.

It should be noted that, after the aperture of the service index is modified, the user can modify the existing index configuration information of the service index by clicking the modification button corresponding to the service index in the configuration page, and resend the modified index configuration information to the application server, so that the application server reenters the modified index configuration information into the corresponding data statistical code to update the data statistical code, and obtains the statistical result corresponding to the aperture of the modified service index by executing the updated data statistical code, thereby avoiding rewriting the code and redeploying, enabling the new adding operation and the modifying operation of the service index to be completed quickly on line, and by importing or modifying the configuration information, timely achieving hot deployment, greatly saving manpower, improving development efficiency, and further, the flexibility and maintainability of service configuration are improved. The user can view specific index configuration information in the configuration page, so that the readability of the code is improved.

According to the technical scheme of the embodiment, the user can configure the service index in the client in advance based on the service index aperture to obtain corresponding index configuration information, so that the client sends the index configuration information to the application server. The application server is preset with configurable data statistical codes, so that when the index configuration information sent by the client is obtained, the index configuration information can be input into the data statistical codes, the dynamic configuration of the codes is realized, and then the statistical results corresponding to the service indexes can be obtained by executing the configured data statistical codes, so that developers do not need to repeatedly write the codes and do not need to re-deploy the codes on line, and only need to configure the codes in the client to obtain the corresponding index configuration information, thereby saving the labor cost and improving the development efficiency.

On the basis of the technical scheme, when the index configuration information comprises: when the time information, the data filtering condition and the data statistical mode are counted, the application server sets corresponding data statistical codes in advance based on the three configuration items so as to input the configuration information corresponding to the three configuration items into the data statistical codes for dynamic configuration, so that the input data statistical codes can realize the function of obtaining the statistical result corresponding to the service index. Illustratively, the input data statistics code may implement a function of obtaining a statistics result corresponding to the service index through the following steps S131 to S133:

s131, determining the current statistical time period according to the statistical time information.

The statistical time information may be used to represent a time requirement corresponding to the service data to be counted. For example, the statistical temporal information may include, but is not limited to: time format and time offset are counted. Wherein the statistical time format may be used to characterize the statistical time level. For example, the statistical time scale may include, but is not limited to, day scale, hour scale, or minute scale. Wherein, the day grade refers to the time grade taking days as a unit; hourly is the time scale in hours; the minute scale refers to a time scale in minutes. The time offset may be used to characterize a statistical start time or a statistical end time as an offset time corresponding to the current system time. For example, the present embodiment may set two time offsets, which are used to respectively represent the offset time when the statistical start time is equivalent to the current system time, and represent the offset time when the statistical end time is equivalent to the current system time; positive and negative values may also be used, for example, a positive value is used to characterize the offset time when the statistical start time corresponds to the current system time, and a negative value is used to characterize the offset time when the statistical end time corresponds to the current system time. The current statistics time period may include a current statistics start time and a current statistics end time, and the current statistics time period may be updated periodically to implement the periodic data statistics.

Exemplarily, S131 may include: determining a statistical time level according to the statistical time format, and determining a statistical time interval according to the statistical time level and the time offset; and determining the current counting starting time and the current counting ending time according to the current system time and the counting time interval.

In particular, the corresponding statistical time level may be determined according to the last accurate time in the statistical time format. For example, if the statistical time format is: determining the statistical time level as a day level according to the year, month and day; if the statistical time format is: determining the statistical time level as an hour level when the year, month and day are reached; if the statistical time format is: and determining the statistic time level to be a minute level by year, month and day. If the time offset is 0, it indicates that there is no time offset, and based on the statistical time scale, the statistical time interval may be determined to be 1 day, 1 hour, or 1 minute. If the time offset is not 0, for example 1, this indicates that a time offset is present, i.e. a statistical time interval of 2 days, 2 hours or 2 minutes can be determined. And if the time offset is used for representing the offset time of the counting starting time, taking the previous time offset of the starting time of the current system time as the current counting starting time, and determining the addition result of the current counting starting time and the counting time interval as the current counting ending time. For example, if the time offset is 1, the time statistic level is in the order of days, and the current system time is 2, 14 and 2020, that is, the starting time of the current system time is 0 min 0s at 2, 14 and 0s at 2020, the offset is one day, that is, 0 min 0s at 13, 0s at 2, 13 and 0s at 2020 can be used as the current statistic starting time, and 59 min 59 s at 23 and 23 at 2, 14 and 14 days at 2020 can be used as the current statistic ending time, so as to count the data of yesterday and that day.

And if the time offset is used for representing the offset time of the counting ending time, taking the starting time of the current system time as the current counting starting time, and determining the addition result of the current counting starting time and the counting time interval as the current counting ending time. For example, if the time offset is 1, the time statistic level is in the order of days, that is, the statistic time interval is 2 days, and the current system time is 14 days 2/month 2020, that is, the starting time of the current system time is 0 min 0 at 0 h 2/14 days 0/2 month 2020, and the ending time is 59 min 59 s at 23 h 2/14 days 23/2020 year, 0 min 0s at 0 h 2/14 days 2020 year may be used as the current statistic starting time, and 59 min 59 s at 23 h 2/15 days 2020 year may be used as the current statistic ending time, so as to count the data of the current day and the next day. It should be noted that, in the present embodiment, the current statistical time period may be periodically updated based on the statistical time interval, so as to obtain the statistical result corresponding to each statistical time period.

S132, filtering the service data in the application database according to the current statistical time period and the data filtering condition, and determining the target service data corresponding to the current statistical time period.

The application database may be a database for storing each service data generated in the docked application system. Different application systems may correspond to different application databases. The target traffic data may refer to traffic data that needs to be counted during the current counting time period.

The data filtering condition can be used to represent the condition that the target service data needs to satisfy. The data filtering condition may include a requirement for statistical time filtering. For example, the data filtering condition corresponding to the service index "business unit volume" may be configured as follows:

“bool”:{“should”:[{“range”:{“store_pack_time”:{“gte”:“@startTime”,“Ite”:“@endTime”}}},{“range”:{“sms_pickup_time”:{“gte”:“@startTime”,“Ite”:“@endTime”}}}],

"must" [ { "term": { "channel _ type": commercially } ],

“must_not”:[{“term”:{“order_status”:“0”}}]

wherein, store _ pack _ time, sms _ pickup _ time, channel _ type and order _ status all refer to the existing fields in the application database. startTime and endTime refer to a current statistics start time and a current statistics end time in a current statistics period, respectively. The data filtering conditions are as follows: when the warehouse packing time store _ pack _ time and the sorting packing time sms _ pickup _ time in a certain piece of service data are within the current statistical time period, the channel type channel _ type is commercial, and the order status order _ status is not in an invalid state, it may be determined that the service data is the target service data corresponding to the current statistical time period.

Specifically, after the current statistical time period is determined, data filtering may be performed on each service data in the current statistical time period based on the data filtering condition, so as to determine the target service data corresponding to the current statistical time period.

And S133, performing data statistics on the target service data based on a data statistics mode, and determining a current statistical result corresponding to the service index.

Specifically, the application server may perform data statistics on the target service data by calling a preset statistical function corresponding to the data statistical manner. If the preset statistical function is a count function, the total number of the target service data corresponding to the current statistical time period can be counted, and the total number is determined as the current statistical result corresponding to the service index. When the preset statistical function is a sum function, the numerical values corresponding to the designated fields in the target service data corresponding to the current statistical time period may be summed, and the summed result is used as the current statistical result corresponding to the service index, for example, the sum of the transaction amount in the target service data is summed, and the obtained summed result is the transaction amount finally required to be counted. In this embodiment, the user can configure the data statistics mode based on the service requirement at the client, so as to meet different statistics requirements of the user.

For example, after the current statistical result corresponding to the service index is determined, the current statistical result may be stored in a database for viewing and displaying. For example, the present embodiment may store the statistical result obtained each time in the form of a key-value pair. The index configuration information may further include a service index identifier for distinguishing different service indexes. In this embodiment, the current statistical service index key information may be determined based on the service index and the statistical time format, the current statistical result may be determined as the current statistical service index value information, and the service index key information and the service index value information may be stored in the database as a key value pair information. Illustratively, the current statistics ending time may be characterized in a statistics time format, a reference time for characterizing the current statistics result is obtained, and the traffic indicator key information is generated based on the traffic indicator identifier and the reference time. For example, the service indicator is identified as BUS _ OPERATOR, when the second statistical end time is 23

days

2 and 14 months 2020, 59 minutes and 59 seconds, and the statistical time format is year, month and day (for example, yyyyMMdd), that is, the statistical time level is day level, the reference time may be determined to be 20200204, and the service indicator key information may be represented as: BUS OPERATOR 20200204 to characterize the statistics obtained on day 14, month 2, 2020.

On the basis of the above technical solution, the index configuration information may further include: the historical statistical result updating parameter, and correspondingly, the data statistical code can also realize the function of updating the historical statistical result based on the historical statistical result updating parameter. For example, after step S133, the following steps may be further included:

and S134, updating parameters according to the historical statistical result, re-determining the target service data corresponding to the previous statistical time period, and performing data statistics to update the previous statistical result corresponding to the previous statistical time period.

Specifically, in some service scenarios, the target service data determined in the current statistical time period may be inaccurate, for example, service data with an expected or time delay may cause the current statistical result to be inaccurate, so that after the current statistical result is counted, the target service data corresponding to the previous statistical time period needs to be re-determined and subjected to data statistics, so as to update the previous statistical result, and improve the accuracy of the historical statistical result. For example, when the transaction amount is counted, if the user performs the order returning operation on the second day after the transaction is successfully performed on the first day, the statistical result on the first day is inaccurate, and when the statistical result corresponding to the second day is determined, the statistical result corresponding to the first day needs to be determined again to improve the accuracy of the historical statistical result. In this embodiment, if the historical statistical result update parameter is 0, it indicates that the historical statistical result does not need to be updated. If the historical statistical result updating parameter is not 0, for example, X, it indicates that X previous historical statistical results need to be updated. For example, when the historical statistical result update parameter is 1, after the current statistical result is determined, that is, before the next statistical result is determined, the statistical update may be performed again on the last statistical time period to update the last statistical result, so as to improve the accuracy of the data statistics.

On the basis of the above technical solution, the index configuration information may further include: historical complement time information. The historical complement time information may include a historical start time and a historical end time for performing complement processing. Illustratively, the historical complement time information completion may be configured to: { "start _ time": 20190305 and "end _ time": 20190307"}, it indicates that the historical data in the specified historical time period of 20190305 to 20190307 needs to be counted to realize the complement function.

Accordingly, the data statistics code may also implement a complement function based on the historical complement time information. Specifically, each complement time period is determined according to the historical complement time information and the statistical time interval, the target historical service data corresponding to each complement time period is determined based on each complement time period and the data filtering condition, data statistics is carried out on the target historical service data based on a data statistics mode, and a statistical result corresponding to each complement time period is determined.

Specifically, the present embodiment generally starts to perform data statistics with reference to the current system time, so that the traffic data in a historical time period before the current system time cannot be counted. However, based on some service requirements, data statistics needs to be performed on a certain historical time period, so that complement processing needs to be performed in order to analyze the statistical result in the historical time period. Illustratively, if the historical complement time information is the historical time period of 20190305 to 20190307, and the statistical time interval is every day, it may be determined that each of three days of 20190305, 20190306, and 20190307 corresponds to one complement time period, so that based on the similar statistical process, a statistical result corresponding to each of the three days may be determined, thereby implementing the complement function of the historical statistical result, further meeting the personalized requirements of the user, and by configuring the historical complement time information on line, it is not necessary to deploy the code on line again, implementing hot deployment, and improving development efficiency.

Example two

Fig. 2 is a flowchart of a current statistical result determining process in a data processing method according to a second embodiment of the present invention. In this embodiment, on the basis of the foregoing embodiment, the index configuration information may further include: and the data grouping field and the corresponding input data statistical code can also realize the function of grouping and counting the service indexes. Wherein explanations of the same or corresponding terms as those of the above-described embodiments are omitted.

Referring to fig. 2, the current statistical result determining process in the data processing method provided in this embodiment specifically includes the following steps:

and S210, determining the current statistical time period according to the statistical time information.

S220, filtering the service data in the application database according to the current statistical time period and the data filtering condition, and determining the target service data corresponding to the current statistical time period.

And S230, grouping the target service data based on the data grouping field, and determining the first service data corresponding to each group.

The data packet field may refer to a certain data field in the application database, which is used as a basis for service data packet, and may be configured based on service requirements. For example, a data packet field may refer to an area field ORG or the like in an application database.

Specifically, the application server may group the target service data based on the data grouping field, so that each group of corresponding first service data corresponds to the same field attribute value. The number of packets is equal to the number of field attribute values corresponding to the data packet fields. For example, when the data grouping field is an area field, if 8 areas are pre-divided, that is, the area field has 8 field attribute values, the first service data corresponding to each group after grouping is the first service data corresponding to the same area.

And S240, performing data statistics on the first service data of each group based on a data statistics mode, and determining a current statistical result corresponding to the service index in each group.

Specifically, data statistics is performed on the first service data of each group based on a data statistics mode, so that a current statistical result corresponding to each group is obtained, and grouping statistics of service indexes is achieved. In the prior art, when statistical results corresponding to different areas need to be obtained, service indexes corresponding to the different areas need to be set so as to obtain a statistical result of a corresponding area based on each service index, thereby reducing statistical efficiency and resource utilization rate.

Illustratively, the index configuration information may further include: and the service index identification and the field identification corresponding to the data packet field. Accordingly, after S240, the method may further include:

determining the corresponding service index key information of the current statistics according to the service index identification, the field identification and the statistics time format in the statistics time information; determining the service index value information corresponding to the current statistics by taking the field attribute value corresponding to each group as sub-key information and taking the current statistical result corresponding to each group as corresponding sub-value information; and storing the service index key information and the service index value information.

Specifically, the current statistics end time may be characterized in a statistics time format, a reference time used for characterizing the current statistics result is obtained, and the service indicator key information corresponding to the current statistics is generated based on the service indicator identifier, the reference time, and the field identifier corresponding to the data packet field. For example, FIG. 3 gives an example of one type of stored key-value pair information. In fig. 3, the service indicator is identified as BUS _ OPERATOR, the field corresponding to the data packet field is identified as ORG, when the secondary counting end time is 59 minutes and 59 seconds when 11/21/00/2019, the counting time format is yearly/monthly (for example, yyyyMMddhh), that is, the counting time level is an hour level, it may be determined that the reference time is 2019112100, and the service indicator key information may be represented as: BUS _ OPERATOR _ ORG _2019112100 to characterize the packet statistics obtained based on packet field ORG at 11/21/2019. The service index value information in this embodiment may be a current statistical result corresponding to each group stored in a key value pair form. In this embodiment, the field attribute value of the data packet field in each group may be used as one piece of sub-key information, the current statistical result corresponding to the group is used as one piece of sub-value information, so as to obtain each piece of sub-key value pair information, and each piece of sub-key value pair information is determined as service index value information of the current statistics, so as to store the current statistical result corresponding to each group in a key value pair form, as shown in fig. 3, the base number in fig. 3 represents the field attribute value, that is, the sub-key information, and the even number represents the current statistical result, that is, the sub-value information, corresponding to the group.

The number of the data grouping fields in this embodiment may be one or more, and if there are multiple data grouping fields, grouping statistics may be performed on a per data grouping field basis one by one, so as to obtain multiple pieces of service index key information and corresponding service index value information.

Illustratively, the index configuration information may further include: a data storage mode, which is configured in a storage mode of the statistical result; accordingly, storing the service index key information and the service index value information may include: and calling a preset storage function corresponding to the data storage mode, and storing the service index key information and the service index value information into a preset database.

The preset database may be a preset database for storing statistical results. For example, the preset database may be a Redis (Remote Dictionary Server) database or a MySQL relational database, or the like. The Redis database is an open-source log-type and Key-Value Key Value pair database which is written by using ANSIC language, supports network and can be based on memory and can also be persistent, so that the storage requirements under different scenes can be met. According to the embodiment, the corresponding preset storage function can be written in advance based on a data storage mode, so that data storage can be rapidly and conveniently carried out in a mode of calling the function in a reflection mode. The data storage mode in this embodiment may be characterized by using a function name of a preset storage function. Illustratively, the data storage mode may be a full storage mode so as to store the statistical result obtained each time, or may be an update storage mode so as to update the existing statistical result based on the currently obtained statistical result, thereby saving the data storage space. When the number of field attribute values corresponding to the data grouping field changes dynamically, that is, when the number of sub-key value pairs in the service index value information changes dynamically, the update storage method may include two update methods, the first method is to directly replace the currently obtained sub-value information (that is, the current statistical result) with the corresponding existing sub-value information (that is, the last statistical result), so as to continuously store the existing sub-value information that is not replaced; and secondly, deleting the existing sub-value information, and then storing the currently obtained sub-value information to delete all the existing sub-value information which is not replaced, so that the diversity of data storage modes is improved, and the personalized requirements of users are further met.

According to the technical scheme of the embodiment, the service indexes are configured with the data grouping fields, so that the grouping statistics of the service indexes can be realized, the statistical results corresponding to different field attribute values under the data grouping fields are simultaneously counted by using one service index, the statistical efficiency and the resource utilization rate are further improved, the personalized requirements of users are further met, and the user experience is improved.

On the basis of the above technical solution, data packet subfields may exist in a nested form in a data packet field so that each group may be further grouped. Exemplarily, after S230, the method may further include: if the data packet sub-field in the nested form exists in the data packet field, grouping the first service data corresponding to each group again based on the data packet sub-field, and determining second service data corresponding to each sub-group; and performing data statistics on each second service data based on a data statistics mode, and determining a current statistical result corresponding to the service index in each subgroup.

The data packet subfield may refer to a field in the application database that further divides the data packet field. For example, the national region may be divided into 8 large regions, and then divided into small regions corresponding to the respective branches in each large region, for example, the data packet field may be ORG, and the corresponding data packet subfield may be ORG _ PRO _ 6. The data packet field in this embodiment may correspond to at least two levels of nesting, and data packet statistics are performed based on the data packet field and the data packet subfields nested at each level, respectively.

Specifically, after target service data is grouped for the first time based on a data grouping field to obtain first service data corresponding to each group, if a data grouping subfield in a nested form exists in the data grouping field, grouping each first service data for the second time based on the data grouping subfield to obtain second service data corresponding to each subgroup and perform data statistics to obtain a current statistical result corresponding to a subfield attribute value corresponding to the data grouping subfield. Similarly, if the data packet subfield contains a nesting subfield, performing third grouping on each second service data based on the subfield to obtain a current statistical result corresponding to the sub-attribute value corresponding to the subfield until completing the packet statistics corresponding to the innermost nested subfield, thereby realizing multi-level nesting statistical grouping and further meeting the personalized requirements of users.

Exemplarily, after obtaining the current statistical result corresponding to each data packet subfield, the service index key information corresponding to the current statistics may be determined according to the statistics time format in the service index identifier, the data packet subfield identifier, and the statistics time information; using the sub-field attribute value corresponding to each sub-group as sub-key information, and using the current statistical result corresponding to each sub-group as corresponding sub-value information, and determining the service index value information corresponding to the current statistics; and calling a preset storage function corresponding to the data storage mode, and storing the service index key information and the service index value information.

Specifically, based on the storage procedure corresponding to the data packet field, the present embodiment may also store the statistical result obtained after grouping based on the data packet subfield in real time. For example, FIG. 4 gives an example of another type of stored key-value pair information. In fig. 4, the service indicator is identified as BUS _ OPERATOR, the field corresponding to the data packet subfield is identified as ORG _ PRO _6, 59 minutes and 59 seconds when the secondary counting end time is 2019, 11, 21, 00, and the counting time format is yearly, monthly and daily (for example, yyyyMMddhh), that is, the counting time level is an hour level, it may be determined that the reference time is 2019112100, and the service indicator key information may be represented as: BUS _ OPERATOR _ ORG _ PRO _6_2019112100, for representing the packet statistic data obtained based on the packet subfield ORG _ PRO at 21/11/2019, as shown in fig. 4, the base sequence number in fig. 4 represents the subfield attribute value, i.e. sub-key information, and the even sequence number represents the current statistic result corresponding to the sub-group, i.e. sub-value information, so that each statistic result obtained by the packet statistics can be stored, and the data storage efficiency is improved.

EXAMPLE III

Fig. 5 is a flowchart of a current statistical result determining process in a data processing method according to a third embodiment of the present invention. In this embodiment, on the basis of the foregoing embodiment, the index configuration information may further include: the data updating time interval and the corresponding input data statistical code can also realize the function of periodically acquiring and updating the current statistical result in the current statistical time period so as to know the change condition of the statistical result in the current statistical time period in real time. Wherein explanations of the same or corresponding terms as those of the above embodiments are omitted.

Referring to fig. 5, the current statistical result determining process in the data processing method provided in this embodiment specifically includes the following steps:

and S510, determining the current statistical time period according to the statistical time information.

S520, according to the current statistical time period and the data filtering condition, filtering the service data in the application database, and determining the target service data corresponding to the current statistical time period.

S530, according to the data updating time interval, in the current statistical time period, periodically executing data statistics based on a data statistical mode to the target service data, and determining the operation of the current statistical result corresponding to the service index.

Wherein the data update time interval may be used to characterize the refresh frequency of the statistical result during the current statistical time period. The data updating time interval is smaller than the statistical time interval corresponding to the current statistical time period. The application server can be preset with a data updating task to realize data updating in the current statistical time period. The scheduling parameter information corresponding to the data updating task can be configured in the client so as to meet different application scenarios. Illustratively, fig. 6 shows an example of the scheduling parameter configuration information corresponding to the data update task, and fig. 6 shows the data update task dispatchTask20s updated every 20 seconds. As shown in fig. 6, the data update task configuration information may include the number of servers, so as to select an appropriate number of servers to execute the task; the service parameter can be used for representing the identification of the data updating task and also can be used for representing the identification of the data updating time interval; 500 indexes are taken every time, 2 indexes are executed every time for representing that at most 500 index configuration information are taken every time, and 2 index configuration information are analyzed every time, so that concurrent execution is realized; execution time rule 5/20 characterizes the start of execution at the 5 th second per minute and is executed every 20 seconds; the thread number is used for representing the number of threads started by executing the task each time; the data interval, the data absence interval, the number of data retries, and the service timeout time are all configuration information for handling abnormal data.

Specifically, the application server may determine each update time point in the current statistical time period based on the data update time interval and/or the execution time rule in the corresponding data update task, determine the target service data corresponding to the current statistical time period when the current system time reaches each update time point, perform data statistics on the target service data based on a data statistics manner, and determine the current statistics result corresponding to the service index. As time goes by, the service data gradually increases, so that the target service data corresponding to the current statistical time period gradually increases, and therefore, the target service data corresponding to the current statistical time period needs to be determined again in each updating, and data statistics is performed based on the currently determined target service data to obtain the currently counted current statistical result.

S540, detecting whether the current system time is less than the current counting end time in the current counting time period, if so, entering the step S550; if not, the process proceeds to step S560.

And S550, updating the existing current statistical result based on the current statistical result.

Specifically, when the current system time is less than the current statistics ending time in the current statistics time period, it indicates that the updating has not ended, and at this time, the existing current statistics result may be updated to the currently obtained current statistics result, thereby implementing dynamic updating of the statistics result. For example, the present embodiment may send the statistical result obtained each time to the client, so as to display the statistical result of the service index on the display interface of the client in real time. For example, if the sales of each day is to be counted, by setting the configuration item of the data update time interval, real-time change of the sales of each period of time in each day can be obtained, for example, the sales of each day gradually increase, so that the user can know the index change in more detail, and further meet the personalized requirements of the user.

And S560, stopping updating, and taking the current statistical result as the final current statistical result corresponding to the service index.

Specifically, when the current system time is equal to the current statistics ending time in the current statistics time period, it indicates that the data updating in the current statistics time period ends, at this time, the updating is stopped, and the currently obtained current statistics result is used as the final current statistics result corresponding to the service index, so as to obtain the final statistics result corresponding to each statistics time period.

According to the technical scheme of the embodiment, data statistics operation can be periodically performed in the current statistics time period based on the configuration item of the data update time interval, so that dynamic update of the statistical result is realized, a user can know the change condition of the service index in the current statistics time period in more detail, and personalized requirements of the user are further met.

Example four

Fig. 7 is a flowchart of a current statistical result determining process in a data processing method according to a fourth embodiment of the present invention. In this embodiment, on the basis of the foregoing embodiment, the index configuration information may further include: the corresponding input data statistics code can also realize the function of obtaining the current statistical result corresponding to the service index by using the specified ES search server. Wherein explanations of the same or corresponding terms as those of the above embodiments are omitted.

Referring to fig. 7, the current statistical result determining process in the data processing method provided in this embodiment specifically includes the following steps:

s610, determining the current statistical time period according to the statistical time information.

S620, determining a data query statistic request according to the current statistic time period, the data filtering condition, the data statistic mode, the ES index information and the ES type information.

The es (elastic search) search server may be a distributed, high-expansion, high-real-time search and data analysis engine, and may also be used for storing data in a distributed manner. Each application system may correspond to one ES search server so as to store service data obtained by the corresponding application system using an ES application database in the ES search server. The ES search server identification may be used to distinguish between different ES search servers. The ES index information may refer to an identification of a database to be searched in the ES search server. The ES type information may refer to table information to be searched in a database to be searched.

Specifically, the application server may perform information assembly based on the current statistical time period, the data filtering condition, the data statistical manner, the ES index information, and the ES type information, and assemble the ES query statement, for example, the current statistical time period and the data filtering condition may be used to assemble a where statement, the data statistical manner may be used to assemble a group statement, and the like, and generate a corresponding data query statistical request based on the ES query statement.

S630, sending the data query statistics request to an ES search server corresponding to the ES search server identification, so that the ES search server determines target service data corresponding to the current statistics time period from an ES application database based on the data query statistics request, and performs data statistics on the target service data to determine a current statistics result corresponding to the service index.

Specifically, the application server may establish communication with the ES search servers corresponding to a plurality of different application systems in advance, so that an index statistical function of different applications may be implemented by using one application server, a platformization may be implemented, and a resource utilization rate may be improved. The application server sends the data query statistics request to the ES search server corresponding to the ES search server identification, so that the ES search server can obtain a corresponding ES query statement by analyzing the data query statistics request, query the target service data corresponding to the current statistics time period from the ES application database by executing the ES query statement, and perform data statistics on the target service data to determine the current statistics result corresponding to the service index. And the ES search server sends the obtained current statistical result to the application server.

It should be noted that, when the statistical time information and the data filtering condition are null information, that is, the statistical time information and the data filtering condition are not configured in the configuration page, information assembly can be directly performed based on the data statistics mode, the ES index information, and the ES type information, and an ES query statement is assembled to meet the data statistics requirement in the service scenario without the time boundary requirement.

When the index configuration information further comprises a data grouping field, a corresponding ES query statement can be generated based on the data grouping field, so that the ES search server obtains a current statistical result corresponding to each group through the ES query statement, and grouping statistics is realized.

And S640, receiving a current statistical result corresponding to the service index sent by the ES search server.

Specifically, the application server may directly obtain the current statistical result corresponding to the service index through the ES search server, so as to simplify the statistical logic of the application server and improve the processing efficiency.

According to the technical scheme of the embodiment, the application server can quickly obtain the statistical result of the service index by utilizing the ES search server, and the data processing efficiency is improved. And the application server can be connected with the ES search servers corresponding to a plurality of different application systems, so that index statistical functions of different applications can be realized by using the application server, the platform is realized, and the resource utilization rate is improved.

The following is an embodiment of a data processing apparatus according to an embodiment of the present invention, which belongs to the same inventive concept as the data processing methods of the above embodiments, and reference may be made to the above embodiments of the data processing method for details that are not described in detail in the embodiments of the data processing apparatus.

EXAMPLE five

Fig. 8 is a schematic structural diagram of a data processing apparatus according to a fifth embodiment of the present invention, which is applicable to a situation of performing data statistics on a service index. The device specifically includes: an index configuration information receiving module 710, an index configuration information input module 720, and a statistical result obtaining module 730.

The index configuration information receiving module 710 is configured to receive index configuration information corresponding to a service index sent by a client, where the index configuration information is obtained by a user through configuration in advance based on a service index caliber; an index configuration information input module 720, configured to input the index configuration information into a preset data statistics code; the statistical result obtaining module 730 is configured to obtain a statistical result corresponding to the service index by executing the input data statistical code.

Optionally, the index configuration information includes: counting time information, data filtering conditions and data counting modes; accordingly, the statistical result obtaining module 730 includes:

a current statistical time period determining unit, configured to determine a current statistical time period according to the statistical time information;

the target service data determining unit is used for filtering the service data in the application database according to the current statistical time period and the data filtering condition, and determining the target service data corresponding to the current statistical time period;

and the current statistical result determining unit is used for performing data statistics on the target service data based on the data statistical mode and determining the current statistical result corresponding to the service index.

Optionally, the statistical time information includes: counting a time format and a time offset; correspondingly, the sub-statistical time period determination unit is specifically configured to: determining a statistic time level according to the statistic time format, and determining a statistic time interval according to the statistic time level and the time offset, wherein the statistic time level comprises a day level, an hour level or a minute level; and determining the current counting starting time and the current counting ending time according to the current system time and the counting time interval.

Optionally, the index configuration information further includes: a data packet field; accordingly, the current statistics determination unit includes:

the first service data determining subunit is used for grouping the target service data based on the data grouping field and determining first service data corresponding to each group;

and the current statistical result determining subunit is used for performing data statistics on the first service data of each group based on a data statistical mode and determining a current statistical result corresponding to the service index in each group.

Optionally, the secondary statistical result determining unit further includes:

a second service data determining unit, configured to, after grouping target service data based on the data grouping field and determining first service data corresponding to each group, if a data grouping subfield in a nested form exists in the data grouping field, perform grouping again on the first service data corresponding to each group based on the data grouping subfield, and determine second service data corresponding to each subgroup;

and the current statistical result determining subunit is also used for performing data statistics on the second service data based on a data statistical mode and determining a current statistical result corresponding to the service index in each subgroup.

Optionally, the index configuration information further includes: the service index identification and the field identification corresponding to the data grouping field; correspondingly, the device also comprises:

the service index key information determining module is used for performing data statistics on each group of first service data based on a data statistics mode, determining a current statistical result corresponding to the service index in each group, and then determining service index key information corresponding to the current statistics according to the service index identification, the field identification and a statistical time format in the statistical time information;

a service index value information determining module, configured to determine service index value information corresponding to current statistics by using the field attribute value corresponding to each group as sub-key information and using the current statistical result corresponding to each group as corresponding sub-value information;

and the information storage module is used for storing the service index key information and the service index value information.

Optionally, the index configuration information further includes: a data storage manner; correspondingly, the information storage module is specifically configured to: and calling a preset storage function corresponding to the data storage mode, and storing the service index key information and the service index value information into a preset database.

Optionally, the index configuration information further includes: a data update time interval; accordingly, the current statistics determination unit is further configured to:

periodically performing data statistics on target service data based on a data statistics mode within a current statistics time period according to a data updating time interval, and determining a current statistics result corresponding to a service index; if the current system time is less than the current counting end time in the current counting time period, updating the existing current counting result based on the current obtained current counting result; if the current system time is equal to the current counting ending time in the current counting time period, stopping updating, and taking the current obtained current counting result as the final current counting result corresponding to the service index; and the data updating time interval is smaller than the statistical time interval corresponding to the current statistical time period.

Optionally, the index configuration information further includes: the ES search server identification, the ES index information and the ES type information; accordingly, the statistical result obtaining module 730 further includes:

the data query statistics request determining unit is used for determining a data query statistics request according to the current statistics time period, the data filtering condition, the data statistics mode, the ES index information and the ES type information;

the data query statistics request sending unit is used for sending the data query statistics request to an ES search server corresponding to the ES search server identifier so that the ES search server determines target service data corresponding to the current statistics time period from an ES application database based on the data query statistics request, and performs data statistics on the target service data to determine a current statistics result corresponding to the service index;

and the current statistical result receiving unit is used for receiving the current statistical result corresponding to the service index sent by the ES search server.

Optionally, the index configuration information further includes: updating parameters according to historical statistical results; correspondingly, the device also comprises:

and the historical statistical result updating module is used for re-determining the target service data corresponding to the last statistical time period according to the historical statistical result updating parameters after determining the current statistical result corresponding to the service index, and performing data statistics to update the last statistical result corresponding to the last statistical time period.

The data processing device provided by the embodiment of the invention can execute the data processing method provided by any embodiment of the invention, and has the corresponding functional module and the beneficial effect of executing the data processing method.

EXAMPLE six

Fig. 9 is a schematic structural diagram of an apparatus according to a sixth embodiment of the present invention. FIG. 9 illustrates a block diagram of an exemplary device 12 suitable for use in implementing embodiments of the present invention. The device 12 shown in fig. 9 is only an example and should not bring any limitation to the function and scope of use of the embodiments of the present invention.

As shown in FIG. 9, device 12 is in the form of a general purpose computing device. The components of device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.

Bus 18 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, micro-channel architecture (MAC) bus, enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

Device 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by device 12 and includes both volatile and nonvolatile media, removable and non-removable media.

The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)30 and/or cache memory 32. Device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 9, and commonly referred to as a "hard drive"). Although not shown in FIG. 9, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.

A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.

Device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), with one or more devices that enable a user to interact with device 12, and/or with any devices (e.g., network card, modem, etc.) that enable device 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Also, the device 12 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet) via the network adapter 20. As shown in FIG. 9, the network adapter 20 communicates with the other modules of the device 12 via the bus 18. It should be appreciated that although not shown in FIG. 9, other hardware and/or software modules may be used in conjunction with device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

The processing unit 16 executes various functional applications and data processing by executing programs stored in the system memory 28, for example, to implement a data processing method provided by the embodiment of the present invention, the method including:

receiving index configuration information corresponding to a service index sent by a client, wherein the index configuration information is obtained by a user through configuration based on the service index caliber in advance;

inputting the index configuration information into a preset data statistical code;

Of course, those skilled in the art will appreciate that the processor may also implement the solution of the method for determining the reserved inventory provided by any embodiment of the present invention.

EXAMPLE seven

The present embodiment provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of a data processing method as provided in any of the embodiments of the invention, the method comprising:

Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer-readable storage medium may be, for example but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

It will be understood by those skilled in the art that the modules or steps of the invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of computing devices, and optionally they may be implemented by program code executable by a computing device, such that it may be stored in a memory device and executed by a computing device, or it may be separately fabricated into various integrated circuit modules, or it may be fabricated by fabricating a plurality of modules or steps thereof into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.

It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data processing method, comprising:

2. The method of claim 1, wherein the metric configuration information comprises: counting time information, data filtering conditions and data counting modes;

correspondingly, the function of obtaining the statistical result corresponding to the service index is realized by the input data statistical code through the following steps:

determining a current statistical time period according to the statistical time information;

filtering the service data in the application database according to the current statistical time period and the data filtering condition, and determining the target service data corresponding to the current statistical time period;

and performing data statistics on the target service data based on the data statistics mode, and determining a current statistical result corresponding to the service index.

3. The method of claim 2, wherein the statistical time information comprises: counting a time format and a time offset;

correspondingly, determining the current statistical time period according to the statistical time information comprises the following steps:

determining a statistic time level according to the statistic time format, and determining a statistic time interval according to the statistic time level and the time offset, wherein the statistic time level comprises a day level, an hour level or a minute level;

and determining the current counting starting time and the current counting ending time according to the current system time and the counting time interval.

4. The method of claim 2, wherein the metric configuration information further comprises: a data packet field;

correspondingly, performing data statistics on the target service data based on the data statistics mode, and determining a current statistical result corresponding to the service index, including:

grouping the target service data based on the data grouping field, and determining first service data corresponding to each group;

and performing data statistics on the first service data of each group based on the data statistics mode, and determining a current statistical result corresponding to the service index in each group.

5. The method of claim 4, further comprising, after grouping the target traffic data based on the data grouping field and determining each group of corresponding first traffic data, further comprising:

if the data packet sub-field in a nested form exists in the data packet field, grouping the first service data corresponding to each group again based on the data packet sub-field, and determining second service data corresponding to each sub-group;

and performing data statistics on the second service data based on the data statistics mode, and determining a current statistical result corresponding to the service index in each subgroup.

6. The method of claim 4, wherein the metric configuration information further comprises: service index identification and field identification corresponding to the data grouping field;

correspondingly, after performing data statistics on each group of the first service data based on the data statistics manner and determining a current statistical result corresponding to the service index in each group, the method further includes:

determining the corresponding service index key information of the current statistics according to the service index identification, the field identification and the statistics time format in the statistics time information;

determining the service index value information corresponding to the current statistics by taking the field attribute value corresponding to each group as sub-key information and taking the current statistical result corresponding to each group as corresponding sub-value information;

and storing the service index key information and the service index value information.

7. The method of claim 6, wherein the metric configuration information further comprises: a data storage manner;

correspondingly, storing the service index key information and the service index value information includes:

and calling a preset storage function corresponding to the data storage mode, and storing the service index key information and the service index value information into a preset database.

8. The method of claim 2, wherein the metric configuration information further comprises: a data update time interval;

according to the data updating time interval, periodically executing data statistics on the target service data based on the data statistics mode in the current statistics time period, and determining the current statistics result corresponding to the service index;

if the current system time is less than the current counting end time in the current counting time period, updating the existing current counting result based on the current obtained current counting result;

if the current system time is equal to the current counting ending time in the current counting time period, stopping updating, and taking the current obtained current counting result as the final current counting result corresponding to the service index;

and the data updating time interval is smaller than the statistical time interval corresponding to the current statistical time period.

9. The method of claim 2, wherein the metric configuration information further comprises: the ES search server identification, the ES index information and the ES type information;

correspondingly, filtering the service data in the application database according to the current statistical time period and the data filtering condition, and determining the target service data corresponding to the current statistical time period; performing data statistics on the target service data based on the data statistics mode, and determining a current statistical result corresponding to the service index, including:

determining a data query statistic request according to the current statistic time period, the data filtering condition, the data statistic mode, the ES index information and the ES type information;

sending the data query statistics request to an ES search server corresponding to the ES search server identification, so that the ES search server determines target service data corresponding to the current statistics time period from an ES application database based on the data query statistics request, and performs data statistics on the target service data to determine a current statistics result corresponding to the service index;

and receiving a current statistical result corresponding to the service index sent by the ES search server.

10. The method according to any one of claims 2-9, wherein the metric configuration information further includes: updating parameters according to historical statistical results;

correspondingly, after determining the current statistical result corresponding to the service index, the method further includes:

and according to the historical statistical result updating parameters, re-determining the target service data corresponding to the last statistical time period and performing data statistics to update the last statistical result corresponding to the last statistical time period.

11. A data processing apparatus, comprising:

12. An apparatus, characterized in that the apparatus comprises:

one or more processors;

a memory for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement a data processing method as claimed in any one of claims 1-10.

13. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the data processing method of any one of claims 1 to 10.