CN117851407A

CN117851407A - Wide-table stability detection method, detection device, storage medium and processor

Info

Publication number: CN117851407A
Application number: CN202311873276.6A
Authority: CN
Inventors: 徐元元; 袁芮; 李少波
Original assignee: Postal Savings Bank of China Ltd
Current assignee: Postal Savings Bank of China Ltd
Priority date: 2023-12-29
Filing date: 2023-12-29
Publication date: 2024-04-09

Abstract

The application provides a detection method, a detection device, a storage medium and a processor for wide-table stability. The method comprises the following steps: acquiring a plurality of pieces of first target data; determining a plurality of second target data according to the plurality of first target data; calculating the average value of all information entropy of the second target data of all the strips to obtain target information entropy; and calculating the difference value between the 1 and the target information entropy to obtain a stable value. The method solves the problems of untimely analysis and strong subjectivity of the existing analysis method for the stability of the broad table.

Description

Wide-table stability detection method, detection device, storage medium and processor

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a method for detecting stability of a broad table, a device for detecting stability of a broad table, a computer readable storage medium, and a processor.

Background

With the great importance of digital transformation in banking industry, the combination of big data application and banking scenes is more compact. The wide table is designed in a high-efficiency model in aspects of service logic, data use and the like, provides powerful support for downstream data application, integrates data of different service topics, is huge in structure, and relates to multiple indexes, corresponding bottom data are derived from various service systems, tables and fields, various dependency relations exist, more performance resources are occupied, and the running speed is slow. In the operation process of the wide table, faults generated in any link are transmitted to the wide table, so that the stability of the wide table is poor, and the characteristic of instability of the wide table is lack of a scientific and effective method for measurement.

The existing analysis method for the stability of the wide table generally analyzes the stability of the wide table when the operation information of the wide table is greatly fluctuated to cause adverse effects, the timeliness cannot be guaranteed, and the log file is simply analyzed and deduced generally by means of personal experience, so that the stability of the wide table is judged, the subjectivity is strong, and the objectivity cannot be guaranteed.

Disclosure of Invention

The main object of the present application is to provide a method for detecting broad-table stability, a device for detecting broad-table stability, a computer readable storage medium and a processor, so as to at least solve the problems of untimely analysis and strong subjectivity of the existing method for analyzing broad-table stability.

In order to achieve the above object, according to one aspect of the present application, there is provided a method for detecting stability of a wide table, the wide table to be detected including a plurality of index fields, the wide table to be detected being run a plurality of times, the method comprising: acquiring a plurality of pieces of first target data, wherein one piece of first target data comprises processing speed types of a plurality of index fields in an operation process, one processing speed type represents the processing speed of one index field, and the wide table to be detected is a wide table of stability to be detected; determining a plurality of pieces of second target data according to the plurality of pieces of first target data, wherein one piece of second target data corresponds to one piece of first target data, and one piece of second target data comprises information entropy of all processing speed types of the corresponding first target data; calculating the average value of all the information entropies of the second target data of all the strips to obtain target information entropies; calculating the difference value between the 1 and the target information entropy to obtain a stable value, wherein the stable value reflects the stability of the wide table to be detected, and the larger the stable value is, the stronger the stability of the wide table to be detected is.

Optionally, in accordance with a plurality of saidThe first target data, and determining a plurality of pieces of second target data, wherein the method further comprises: according to item i, the first target data and E _ij ＝-P _ij log ₂ P _ij Determining the information entropy of the processing speed type j during the ith operation, wherein E _ij For the information entropy, P, of the process speed type j during the ith run _ij Is a ratio of the number of the processing speed types j contained in the first target data and the total number of the processing speed types contained in the first target data.

Optionally, acquiring a plurality of first target data includes: acquiring a plurality of pieces of third target data, wherein one piece of third target data comprises processing time lengths of a plurality of index fields in one operation process; and determining a plurality of pieces of first target data according to the third target data and a plurality of preset mapping relations, wherein one index field corresponds to one preset mapping relation, and the preset mapping relation corresponding to the index field is a mapping relation between the processing time length of the index field and the processing speed type of the index field.

Optionally, one of the running processes includes a plurality of processing processes, wherein a plurality of the index fields are processed in the processing process to obtain a plurality of third target data, including: acquiring names of a plurality of index fields from the wide table to be detected; according to the names of all the index fields, acquiring a plurality of pieces of fourth target data from a log file, wherein one piece of fourth target data at least comprises processing time length information of each index field in the running process, the processing time length information of each index field comprises the time length of the processing process of the index field and the processing time length of a dependence table of the index field, and the dependence table is a wide table used for processing the index field; and determining a plurality of pieces of third target data according to all pieces of fourth target data, wherein the processing time length of one index field is the sum of the time length of the processing process of the index field and the processing time length of the dependency table of the index field.

Optionally, one piece of the fourth target data further includes name information of each of the index fields in one of the running processes, the name information including a name of the index field, a name of the machining process in which the index field is located, and a name of the dependency table of the index field, after acquiring a plurality of pieces of the fourth target data from a log file according to all the names of the index fields, before determining a plurality of pieces of the third target data according to all the pieces of the fourth target data, the method further includes: determining, according to the name information of a first target index field, whether fifth target data has the name information of a second target index field, where the first target index field is the name of any one of the index fields included in the fifth target data, the fifth target data is any piece of the fourth target data, the second target index field is the index field different from the name of the second target index field in the fifth target data, the name of the machining process where the second target index field is located is the same as the name of the machining process where the first target index field is located, and the name of the dependency table of the second target index field is the same as the name of the dependency table of the first target index field; a deleting step of deleting, in a case where the name information of the second target index field exists in the fifth target data, the name information of the second target index field in the fifth target data and the processing time length information of the second target index field in the fifth target data; a first repeating step of repeating the determining step and the deleting step at least once until the fifth target data does not have the name information of the second target index field; and a second repeating step, wherein the first repeating step is repeated at least once until all the processing work of the fourth target data is finished.

Optionally, acquiring names of a plurality of the indicator fields from the wide table to be detected includes: acquiring names of a plurality of fields and types of the fields from the log file; in the case that the type of the field is a number type, an int type, or a decmal type, the name of the field is determined as the name of the index field.

Optionally, the method comprises: acquiring the stable values of a plurality of wide tables to be detected; sequencing according to the sequence from small to large of the stable values to obtain a stable value sequence; generating alarm information according to names of the wide tables to be detected corresponding to the M top stable values in the stable value sequence, wherein the alarm information indicates that the wide tables to be detected are unstable.

According to another aspect of the present application, there is provided a wide table stability detection apparatus, where a wide table to be detected includes a plurality of index fields, and the wide table to be detected runs a plurality of times, the apparatus includes: the first acquisition unit is used for acquiring a plurality of pieces of first target data, wherein one piece of first target data comprises processing speed types of a plurality of index fields in an operation process, one processing speed type represents the processing speed of one index field, and the wide table to be detected is a wide table of stability to be detected; the first determining unit is used for determining a plurality of pieces of second target data according to the plurality of pieces of first target data, wherein one piece of second target data corresponds to one piece of first target data, and one piece of second target data comprises information entropy of all processing speed types of the corresponding first target data; the first calculation unit is used for calculating the average value of all the information entropies of the second target data of all the strips to obtain target information entropies; the second calculation unit is used for calculating the difference value between the target information entropy and the target information entropy 1 to obtain a stable value, the stable value reflects the stability of the wide table to be detected, and the larger the stable value is, the stronger the stability of the wide table to be detected is.

According to still another aspect of the present application, there is provided a computer readable storage medium, where the computer readable storage medium includes a stored program, and when the program runs, the apparatus in which the computer readable storage medium is controlled to execute any one of the methods for detecting broad-table stability.

According to yet another aspect of the present application, there is provided a processor for running a program, wherein the program runs on executing any one of the broad table stability detection methods.

By the aid of the technical scheme, the processing speed types of the index fields are obtained in the running processes of the wide table to be detected, the stable value reflecting the stability of the wide table to be detected is obtained through calculation according to the processing speed types of the index fields in the running processes of the wide table to be detected, compared with the existing analysis method of the stability of the wide table, the stability of the wide table is analyzed generally when the running information of the wide table is greatly fluctuated to cause adverse effects, timeliness cannot be guaranteed, the stability of the wide table to be detected can be detected in real time in the running process of the wide table to be detected, timeliness of the stability detection of the wide table is guaranteed, the existing analysis method of the stability of the wide table generally conducts simple analysis deduction on a log file according to personal experience, the stability of the wide table is judged, subjectivity is relatively strong, the processing speed types of the index fields in the running processes of the wide table to be detected are automatically determined to reflect the stability of the wide table to be detected, subjective analysis of the stability of the wide table to be detected is not needed, the existing subjective analysis of the stability of the wide table to be detected is guaranteed, and the existing analysis method of the stability of the wide table to be detected is not needed is guaranteed.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:

fig. 1 is a block diagram showing a hardware configuration of a mobile terminal according to a method of detecting wide table stability provided in an embodiment of the present application;

fig. 2 is a schematic flow chart of a method for detecting broad table stability according to an embodiment of the present application;

fig. 3 shows a block diagram of a detection apparatus for broad table stability according to an embodiment of the present application.

Detailed Description

It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

In order to make the present application solution better understood by those skilled in the art, the following description will be made in detail and with reference to the accompanying drawings in the embodiments of the present application, it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, shall fall within the scope of the present application.

It should be noted that the terms "first," "second," and the like in the description and claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate in order to describe the embodiments of the present application described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

For convenience of description, the following will describe some terms or terms related to the embodiments of the present application:

broad table: the method refers to a database table in which dimensions, attributes and indexes related to a service entity are associated together, so that unified storage of different information among the service entities is realized.

Index field: the quantized measurement value obtained by subdividing the business unit in the wide table is usually obtained by aggregation calculation modes such as addition, average and the like, and is an important basis of quantization effect.

As introduced in the background art, the existing analysis method of the wide-table stability has the problems of untimely analysis and strong subjectivity, and in order to solve the problems of untimely analysis and strong subjectivity of the existing analysis method of the wide-table stability, the embodiment of the application provides a detection method of the wide-table stability, a detection device of the wide-table stability, a computer-readable storage medium and a processor.

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

The method embodiments provided in the embodiments of the present application may be performed in a mobile terminal, a computer terminal or similar computing device. Taking the mobile terminal as an example, fig. 1 is a block diagram of a hardware structure of the mobile terminal according to a method for detecting the stability of a broad table according to an embodiment of the present invention. As shown in fig. 1, a mobile terminal may include one or more (only one is shown in fig. 1) processors 102 (the processor 102 may include, but is not limited to, a microprocessor MCU or a processing device such as a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 for communication functions and an input-output device 108. It will be appreciated by those skilled in the art that the structure shown in fig. 1 is merely illustrative and not limiting of the structure of the mobile terminal described above. For example, the mobile terminal may also include more or fewer components than shown in fig. 1, or have a different configuration than shown in fig. 1.

The memory 104 may be used to store a computer program, for example, a software program of application software and a module, such as a computer program corresponding to a display method of device information in an embodiment of the present invention, and the processor 102 executes the computer program stored in the memory 104 to perform various functional applications and data processing, that is, to implement the above-described method. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory remotely located relative to the processor 102, which may be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof. The transmission means 106 is arranged to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, simply referred to as NIC) that can connect to other network devices through a base station to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used to communicate with the internet wirelessly.

In the present embodiment, a method for detecting the stability of a broad table operating on a mobile terminal, a computer terminal or similar computing device is provided, it should be noted that the steps illustrated in the flowchart of the drawings may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order different from that herein.

The wide table to be detected includes a plurality of index fields, and the wide table to be detected runs multiple times, and fig. 2 is a flowchart of a method for detecting stability of the wide table according to an embodiment of the present application. As shown in fig. 2, the method comprises the steps of:

step S201, obtaining a plurality of pieces of first target data;

wherein, one piece of the first target data comprises processing speed types of a plurality of index fields in the operation process, one processing speed type represents the processing speed of one index field, and the wide table to be detected is a wide table of stability to be detected;

specifically, the wide table to be detected is run 5 to 10 times, in an alternative embodiment, as shown in table 1, the wide table to be detected is run 8 times, one sequence number in a sequence number list represents one index field, and processing speed types of a plurality of index fields are obtained in a plurality of running processes of the wide table to be detected, wherein the processing speed types comprise: fast (indicated by 1), medium fast (indicated by 2), normal speed (indicated by 3), medium slow (indicated by 4), slow (indicated by 5).

TABLE 1

In an alternative embodiment, the step S201 may be implemented as:

step S2011, obtaining a plurality of pieces of third target data, wherein one piece of third target data comprises processing time lengths of a plurality of index fields in one running process;

one of the above-described operations includes a plurality of processes in which a plurality of the above-described index fields are processed, and in an alternative embodiment, the above-described step S2011 may be implemented as:

step S20111, obtaining the names of a plurality of index fields from the wide table to be detected;

in an alternative embodiment, the step S20111 may be implemented as follows:

acquiring names of a plurality of fields and types of the fields from the log file;

in the case that the type of the above field is a number type, an int type, or a decmal type, the name of the above field is determined as the name of the above index field.

Specifically, in some alternative embodiments, as shown in table 2, the log file includes: the name of the wide table to be detected, the names of a plurality of fields and the types of the fields, the number type field, the int type field and the decimal type field are all index fields.

TABLE 2

Name of wide table to be detected	Name of field	Type of field
			table_a	field_1	varchar
table_a	field_2	date
			table_a	field_3	number
table_a	field_4	varchar
			table_a	field_5	int
table_a	field_6	decimal
			table_a	field_7	int

Step S20112, obtaining a plurality of pieces of fourth target data from a log file according to the names of all the index fields, wherein one piece of fourth target data at least comprises processing time length information of each index field in the running process, the processing time length information of the index field comprises the time length of the processing process of the index field and the processing time length of a dependence table of the index field, and the dependence table is a wide table used for processing the index field;

specifically, in some alternative embodiments, as shown in table 3, the log file includes: the name of the wide table to be detected, the name of the index field, the processing order (the order of the processing procedures), the name of the processing procedure, the duration of the processing procedure, the name of the dependency table, and the processing duration of the dependency table.

TABLE 3 Table 3

Step S20113, determining a plurality of pieces of third target data according to all pieces of the fourth target data, where the processing duration of one of the index fields is the sum of the duration of the processing procedure in which the index field is located and the processing duration of the dependency table of the index field.

Specifically, when the processing time of the index field is calculated, the time of the processing process of the index field and the processing time of the dependency table of the index field are considered, and the accuracy of the stability detection of the wide table is ensured.

The fourth object data further includes name information of each of the index fields in one of the running processes, the name information including a name of the index field, a name of the machining process in which the index field is located, and a name of the dependency table of the index field, and in an optional embodiment, after the step S2012, before the step S20113, the method further includes:

a determining step of determining, based on the name information of a first target index field, whether or not there is the name information of a second target index field, the first target index field being a name of any one of the index fields included in the fifth target data, the fifth target data being any one of the fourth target data, the second target index field being the index field of the fifth target data that is different from the second target index field, and a name of the machining process in which the second target index field is located being the same as a name of the machining process in which the first target index field is located, and a name of the dependency table of the second target index field being the same as a name of the dependency table of the first target index field;

A deleting step of deleting, when the name information of the second target index field exists in the fifth target data, the name information of the second target index field in the fifth target data and the processing time length information of the second target index field in the fifth target data;

a first repeating step of repeating the determining step and the deleting step at least once until the fifth target data does not have the name information of the second target index field;

and a second repeating step of repeating the first repeating step at least once until all the processing of the fourth target data is completed.

Specifically, if the processing procedures of the index fields are the same and the dependency tables of the index fields are the same, only any one of the index fields is taken as a statistics item, and as shown in table 3, only one index field field_1 and index field_2 are counted, and only one index field field_4 is counted.

Step 2012, determining a plurality of pieces of first target data according to each piece of third target data and a plurality of preset mapping relations, wherein one of the index fields corresponds to one of the preset mapping relations, and the preset mapping relation corresponding to the index field is a mapping relation between a processing duration of the index field and the processing speed type of the index field.

Specifically, in some optional embodiments, as shown in table 4, the name of the wide table to be detected is table_a, the name of the index field is field_1, and the preset mapping relationship corresponding to the index field field_1 is: the process time of the index field field_1 is respectively 5min,10min,13 min and 16min, the process time of different index fields is different, and therefore, the process time threshold of each index field is different in the mapping relation.

TABLE 4 Table 4

Step S202, determining a plurality of pieces of second target data according to the plurality of pieces of first target data;

wherein one piece of the second target data corresponds to one piece of the first target data, and one piece of the second target data includes information entropy of all the processing speed types of the corresponding first target data;

in an alternative embodiment, the step S202 may be implemented as:

according to item i, the first target data and E _ij ＝-P _ij log ₂ P _ij Determining the entropy of the processing speed type j during the ith operation, wherein E _ij For the information entropy, P, of the processing speed type j during the ith operation _ij The ratio of the number of the machining speed types j included in the first target data to the total number of the machining speed types included in the first target data is the i-th item.

Specifically, as shown in table 5, information entropy of the type of processing speed (1, 2, 3, 4, and 5) in the 1 st run, the 2 nd run, the 3 rd run, the 4 th run, the 5 th run, the 6 th run, the 7 th run, the 8 th run was obtained.

TABLE 5

Step S203, calculating the average value of all the information entropies of the second target data of all the strips to obtain a target information entropy;

in particular according toDetermining a sum of information entropy of the type j of the processing speed in the ith operation, wherein E _i The sum of the information entropy of the type j of processing speed in the ith run is determined by +.>The average value of the information entropy of the type (1, 2, 3, 4 and 5) of the processing speed in the 1 st operation, the 2 nd operation, the 3 rd operation, the 4 th operation, the 5 th operation, the 6 th operation, the 7 th operation and the 8 th operation is determined, and as shown in table 5, the target information entropy is 0.6374.

Step S204, calculating the difference value between the 1 and the target information entropy to obtain a stable value;

wherein the stable value reflects the stability of the wide table to be detected, and a larger stable value indicates a stronger stability of the wide table to be detected.

Specifically, the target information entropy before the wide table to be detected is 0.6374, the stable value is 0.3626, as shown in table 6, the target information entropy after the wide table to be detected is 0.5469, the stable value is 0.4531, that is, the optimized stable value is larger, and the optimized wide table is more stable.

TABLE 6

In an alternative embodiment, the method includes:

acquiring the stable values of a plurality of wide tables to be detected;

sequencing according to the sequence from small to large of the stable values to obtain a stable value sequence;

generating alarm information according to the names of the wide table to be detected corresponding to the M top stable values in the stable value sequence, wherein the alarm information indicates that the wide table to be detected is unstable.

Specifically, calculating stable values of each wide table in the wide surface layer, arranging the stable values in sequence from small to large, and taking names of the wide tables corresponding to the first 30 stable values, wherein the wide tables corresponding to the 30 stable values are unstable and need to be optimized.

According to the embodiment, the processing speed types of the plurality of index fields are obtained in the plurality of running processes of the wide table to be detected, the stable value reflecting the stability of the wide table to be detected is obtained through calculation according to the processing speed types of the plurality of index fields in the plurality of running processes of the wide table to be detected, compared with the existing analysis method of the stability of the wide table, the stability of the wide table is analyzed generally when the large fluctuation of the running information of the wide table causes adverse effects, timeliness cannot be guaranteed, the stability of the wide table to be detected can be detected in real time in the running process of the wide table to be detected, timeliness of the stability detection of the wide table is guaranteed, the existing analysis method of the stability of the wide table generally carries out simple analysis deduction on a log file according to personal experience, the stability of the wide table is judged, subjectivity is relatively strong, the processing speed types of the plurality of index fields in the running processes of the wide table to be detected are automatically determined to reflect the stable value of the wide table to be detected, manual analysis is not needed, the existing subjective analysis method of the stability of the wide table to be detected is guaranteed, and the existing analysis of the stability of the wide table is not required to be carried out in time.

In order to enable those skilled in the art to more clearly understand the technical solutions of the present application, the implementation process of the method for detecting the broad-table stability of the present application will be described in detail below with reference to specific embodiments.

The embodiment relates to a specific method for detecting the stability of a broad table, which comprises the following steps:

step S1: acquiring names of a plurality of fields and types of the fields from the log file;

step S2: in the case that the type of the field is a number type, an int type or a decmal type, determining that the name of the field is the name of the index field;

step S3: according to the names of all the index fields, acquiring a plurality of pieces of fourth target data from a log file, wherein one piece of fourth target data at least comprises processing time length information of each index field in the running process, the processing time length information of the index field comprises the time length of the processing process of the index field and the processing time length of a dependence table of the index field, and the dependence table is a wide table used for processing the index field;

step S4: a determining step of determining, based on the name information of a first target index field, whether or not there is the name information of a second target index field, the first target index field being a name of any one of the index fields included in the fifth target data, the fifth target data being any one of the fourth target data, the second target index field being the index field of the fifth target data that is different from the second target index field, and a name of the machining process in which the second target index field is located being the same as a name of the machining process in which the first target index field is located, and a name of the dependency table of the second target index field being the same as a name of the dependency table of the first target index field;

Step S5: a deleting step of deleting, when the name information of the second target index field exists in the fifth target data, the name information of the second target index field in the fifth target data and the processing time length information of the second target index field in the fifth target data;

step S6: a first repeating step of repeating the determining step and the deleting step at least once until the fifth target data does not have the name information of the second target index field;

step S7: a second repeating step of repeating the first repeating step at least once until all the processing operations of the fourth target data are completed;

step S8: determining a plurality of pieces of third target data according to all pieces of fourth target data, wherein the processing time length of one index field is the sum of the time length of the processing process of the index field and the processing time length of the dependency table of the index field;

step S9: determining a plurality of pieces of first target data according to the third target data and a plurality of preset mapping relations, wherein one index field corresponds to one preset mapping relation, and the preset mapping relation corresponding to the index field is a mapping relation between the processing time length of the index field and the processing speed type of the index field;

Step S10: determining a plurality of pieces of second target data according to the plurality of pieces of first target data, wherein one piece of second target data corresponds to one piece of first target data, and one piece of second target data comprises information entropy of all processing speed types of the corresponding first target data;

step S11: calculating the average value of all the information entropies of all the second target data to obtain target information entropies;

step S12: calculating the difference between the entropy of the target information and the entropy of the target information to obtain a stable value, wherein the stable value reflects the stability of the wide table to be detected, and the larger the stable value is, the stronger the stability of the wide table to be detected is.

It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is illustrated in the flowcharts, in some cases the steps illustrated or described may be performed in an order other than that illustrated herein.

The embodiment of the application also provides a device for detecting the stability of the wide table, and the device for detecting the stability of the wide table can be used for executing the method for detecting the stability of the wide table. The device is used for realizing the above embodiments and preferred embodiments, and is not described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

The following describes a wide-table stability detection device provided in the embodiment of the present application.

Fig. 3 is a schematic diagram of a broad table stability detection device according to an embodiment of the present application. As shown in fig. 3, the apparatus includes:

a first acquisition unit 10 for acquiring a plurality of pieces of first target data;

TABLE 1

In an alternative embodiment, the first obtaining unit includes:

an obtaining subunit, configured to obtain a plurality of third target data, where one piece of third target data includes processing durations of a plurality of the index fields in one of the running processes;

One of the runs includes a plurality of processes in which a plurality of the index fields are processed, and in an alternative embodiment, the acquisition subunit includes:

the first acquisition module is used for acquiring names of a plurality of index fields from the wide table to be detected;

in an alternative embodiment, the step S20111 may be implemented as follows:

TABLE 2

The second obtaining module is configured to obtain a plurality of pieces of fourth target data from a log file according to names of all the index fields, where one piece of fourth target data at least includes processing duration information of each index field in the running process, where the processing duration information of the index field includes a duration of the processing process where the index field is located and a processing duration of a dependency table of the index field, where the dependency table is a wide table used for processing the index field;

TABLE 3 Table 3

And the determining module is used for determining a plurality of pieces of third target data according to all pieces of fourth target data, wherein the processing time length of one index field is the sum of the time length of the processing process where the index field is located and the processing time length of the dependency table of the index field.

The fourth object data further includes name information of each of the index fields in one of the running processes, the name information including a name of the index field, a name of the process in which the index field is located, and a name of the dependency table of the index field, and in an alternative embodiment, the apparatus further includes:

A second determining unit configured to determine, based on the name information of a first target index field, whether or not there is the name information of a second target index field in fifth target data, the first target index field being a name of any one of the index fields included in the fifth target data, the fifth target data being any one of the fourth target data, the second target index field being the index field of the fifth target data that is different from the second target index field, a name of the machining process in which the second target index field is located being the same as a name of the machining process in which the first target index field is located, and a name of the dependency table of the second target index field being the same as a name of the dependency table of the first target index field;

a deleting unit configured to delete, when the name information of the second target index field exists in the fifth target data, the name information of the second target index field in the fifth target data and the processing time length information of the second target index field in the fifth target data;

A first repeating unit configured to repeat the determining step and the deleting step at least once until the fifth target data does not have the name information of the second target index field;

and a second repeating unit configured to repeat the first repeating step at least once until all the processing operations of the fourth target data are completed.

The determining subunit is configured to determine, according to each piece of third target data and a plurality of preset mapping relationships, a plurality of pieces of first target data, one of the index fields corresponds to one of the preset mapping relationships, and the preset mapping relationship corresponding to the index field is a mapping relationship between a processing duration of the index field and the processing speed type of the index field.

TABLE 4 Table 4

A first determining unit 20 for determining a plurality of pieces of second target data based on the plurality of pieces of first target data;

in an alternative embodiment, the first determining unit is configured to:

TABLE 5

A first calculation unit 30 for calculating an average value of all the information entropies of the second target data of all the pieces to obtain a target information entropy;

A second calculating unit 40, configured to calculate a difference between 1 and the target information entropy to obtain a stable value;

TABLE 6

In an alternative embodiment, the method includes:

the second acquisition unit is used for acquiring the stable values of the plurality of wide tables to be detected;

the sorting unit is used for sorting the stable values in order from small to large to obtain a stable value sequence;

and the alarm unit is used for generating alarm information according to the names of the wide table to be detected corresponding to the M top stable values in the stable value sequence, wherein the alarm information indicates that the wide table to be detected is unstable.

The device for detecting the stability of the broad table comprises a processor and a memory, wherein the first acquisition unit, the first determination unit, the first calculation unit, the second calculation unit and the like are all stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions. The modules are all located in the same processor; alternatively, the above modules may be located in different processors in any combination.

The processor includes a kernel, and the kernel fetches the corresponding program unit from the memory. The kernel can be provided with one or more than one, and the problems of untimely analysis and strong subjectivity of the existing analysis method for the stability of the broad table are solved by adjusting kernel parameters.

The memory may include volatile memory, random Access Memory (RAM), and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM), among other forms in computer readable media, the memory including at least one memory chip.

The embodiment of the invention provides a computer readable storage medium, which comprises a stored program, wherein the program is controlled to control a device where the computer readable storage medium is located to execute the method for detecting the stability of the broad table.

Specifically, the method for detecting the stability of the broad table comprises the following steps:

step S201, obtaining a plurality of pieces of first target data;

wherein the stability value reflects the stability of the wide table to be detected, and a larger stability value indicates a stronger stability of the wide table to be detected.

The embodiment of the invention provides a processor, which is used for running a program, wherein the method for detecting the stability of a wide table is executed when the program runs.

step S201, obtaining a plurality of pieces of first target data;

The embodiment of the invention provides equipment, which comprises a processor, a memory and a program stored in the memory and capable of running on the processor, wherein the processor realizes at least the following steps when executing the program:

Step S201, obtaining a plurality of pieces of first target data;

The device herein may be a server, PC, PAD, cell phone, etc.

The present application also provides a computer program product adapted to perform a program initialized with at least the following method steps when executed on a data processing device:

Step S201, obtaining a plurality of pieces of first target data;

It will be appreciated by those skilled in the art that the modules or steps of the invention described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may be implemented in program code executable by computing devices, so that they may be stored in a storage device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than that shown or described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, etc., such as Read Only Memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises an element.

From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

1) According to the method for detecting the stability of the wide table, the processing speed types of the index fields are obtained in the multiple operation processes of the wide table to be detected, the stable value reflecting the stability of the wide table to be detected is obtained through calculation according to the processing speed types of the index fields in the multiple operation processes of the wide table to be detected, compared with the existing analysis method for the stability of the wide table, the stability of the wide table is analyzed generally when the large fluctuation of the operation information of the wide table causes bad influence, timeliness cannot be ensured, the stability of the wide table to be detected can be detected in real time in the operation processes of the wide table to be detected, timeliness of the stability detection of the wide table is ensured, the existing analysis method for the stability of the wide table generally carries out simple analysis deduction on a log file by means of personal experience, the stability of the wide table is judged, subjectivity is relatively strong, objectivity cannot be ensured, the stability value reflecting the wide table to be detected is automatically determined, manual analysis is not required, the objectivity of the stability detection of the wide table is ensured, and the existing subjective analysis method for the stability of the wide table is not analyzed in time is solved.

2) In the wide table stability detection device, processing speed types of a plurality of index fields are acquired in a plurality of operation processes of a wide table to be detected, a stable value reflecting the stability of the wide table to be detected is calculated according to the processing speed types of the plurality of index fields in the plurality of operation processes of the wide table to be detected, compared with the existing wide table stability analysis method, the wide table stability is generally analyzed when the large fluctuation of the wide table operation information causes bad influence, timeliness cannot be ensured, the wide table stability to be detected can be detected in real time in the operation processes of the wide table to be detected, timeliness of the wide table stability detection is ensured, the existing wide table stability analysis method generally relies on personal experience to conduct simple analysis deduction on a log file, further judgment on the wide table stability is relatively strong in subjectivity, objectivity cannot be ensured, the stability value reflecting the wide table to be detected is automatically determined, manual analysis is not required, the objectivity of the wide table stability detection is ensured, and the existing subjective stability analysis method is not suitable for the wide table stability detection.

The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the same, but rather, various modifications and variations may be made by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for detecting stability of a broad table, wherein the broad table to be detected includes a plurality of index fields, and the broad table to be detected is operated a plurality of times, the method comprising:

acquiring a plurality of pieces of first target data, wherein one piece of first target data comprises processing speed types of a plurality of index fields in an operation process, one processing speed type represents the processing speed of one index field, and the wide table to be detected is a wide table of stability to be detected;

determining a plurality of pieces of second target data according to the plurality of pieces of first target data, wherein one piece of second target data corresponds to one piece of first target data, and one piece of second target data comprises information entropy of all processing speed types of the corresponding first target data;

calculating the average value of all the information entropies of the second target data of all the strips to obtain target information entropies;

Calculating the difference value between the 1 and the target information entropy to obtain a stable value, wherein the stable value reflects the stability of the wide table to be detected, and the larger the stable value is, the stronger the stability of the wide table to be detected is.

2. The method of claim 1, wherein in determining a plurality of second target data from a plurality of the first target data, the method further comprises:

according to item i, the first target data and E _ij ＝-P _ij log ₂ P _ij Determining the information entropy of the processing speed type j in the ith operation process, wherein E _ij For the information entropy of the process speed type j during the ith operation,P _ij is a ratio of the number of the processing speed types j contained in the first target data and the total number of the processing speed types contained in the first target data.

3. The method of claim 1, wherein obtaining a plurality of first target data comprises:

acquiring a plurality of pieces of third target data, wherein one piece of third target data comprises processing time lengths of a plurality of index fields in one operation process;

and determining a plurality of pieces of first target data according to the third target data and a plurality of preset mapping relations, wherein one index field corresponds to one preset mapping relation, and the preset mapping relation corresponding to the index field is a mapping relation between the processing time length of the index field and the processing speed type of the index field.

4. A method according to claim 3, wherein one of said runs comprises a plurality of processes, said processes processing a plurality of said index fields to obtain a plurality of third target data, comprising:

acquiring names of a plurality of index fields from the wide table to be detected;

according to the names of all the index fields, acquiring a plurality of pieces of fourth target data from a log file, wherein one piece of fourth target data at least comprises processing time length information of each index field in the running process, the processing time length information of each index field comprises the time length of the processing process of the index field and the processing time length of a dependence table of the index field, and the dependence table is a wide table used for processing the index field;

and determining a plurality of pieces of third target data according to all pieces of fourth target data, wherein the processing time length of one index field is the sum of the time length of the processing process of the index field and the processing time length of the dependency table of the index field.

5. The method of claim 4, wherein one piece of the fourth target data further includes name information of each of the index fields in one of the runs, the name information including a name of the index field, a name of the process in which the index field is located, and a name of the dependency table of the index field, and after a plurality of pieces of the fourth target data are acquired from a log file based on the names of all of the index fields, before a plurality of pieces of the third target data are determined based on all of the pieces of the fourth target data, the method further comprises:

Determining whether fifth target data has the name information of a second target index field according to the name information of a first target index field, wherein the first target index field is the name of any one index field contained in the fifth target data, the fifth target data is any piece of fourth target data, the second target index field is the index field which is different from the second target index field in the fifth target data, the name of the machining process where the second target index field is located is the same as the name of the machining process where the first target index field is located, and the name of the dependency table of the second target index field is the same as the name of the dependency table of the first target index field;

a deleting step of deleting, in a case where the name information of the second target index field exists in the fifth target data, the name information of the second target index field in the fifth target data and the processing time length information of the second target index field in the fifth target data;

and a second repeating step, wherein the first repeating step is repeated at least once until all the processing work of the fourth target data is finished.

6. The method of claim 4, wherein obtaining names of a plurality of the indicator fields from the wide table to be detected comprises:

in the case that the type of the field is a number type, an int type, or a decmal type, the name of the field is determined as the name of the index field.

7. The method according to claim 1, characterized in that the method comprises:

acquiring the stable values of a plurality of wide tables to be detected;

generating alarm information according to names of the wide tables to be detected corresponding to the M top stable values in the stable value sequence, wherein the alarm information indicates that the wide tables to be detected are unstable.

8. A wide table stability detection apparatus, wherein a wide table to be detected includes a plurality of index fields, the wide table to be detected is run a plurality of times, the apparatus comprising:

the first acquisition unit is used for acquiring a plurality of pieces of first target data, wherein one piece of first target data comprises processing speed types of a plurality of index fields in an operation process, one processing speed type represents the processing speed of one index field, and the wide table to be detected is a wide table of stability to be detected;

the first determining unit is used for determining a plurality of pieces of second target data according to the plurality of pieces of first target data, wherein one piece of second target data corresponds to one piece of first target data, and one piece of second target data comprises information entropy of all processing speed types of the corresponding first target data;

the first calculation unit is used for calculating the average value of all the information entropies of the second target data of all the strips to obtain target information entropies;

the second calculation unit is used for calculating the difference value between the target information entropy and the target information entropy 1 to obtain a stable value, the stable value reflects the stability of the wide table to be detected, and the larger the stable value is, the stronger the stability of the wide table to be detected is.

9. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a stored program, wherein the program, when run, controls a device in which the computer-readable storage medium is located to perform the method for detecting broad-table stability according to any one of claims 1 to 7.

10. A processor, characterized in that the processor is configured to run a program, wherein the program, when run, performs the method for detecting broad table stability according to any of claims 1 to 7.