CN115981964A - Data detection method and device - Google Patents
Data detection method and device Download PDFInfo
- Publication number
- CN115981964A CN115981964A CN202310007562.8A CN202310007562A CN115981964A CN 115981964 A CN115981964 A CN 115981964A CN 202310007562 A CN202310007562 A CN 202310007562A CN 115981964 A CN115981964 A CN 115981964A
- Authority
- CN
- China
- Prior art keywords
- data
- detection
- detected
- information
- analyzed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a data detection method and a device, wherein the data detection method comprises the following steps: acquiring data to be detected in a current detection period, and determining data detection information of the data to be detected, wherein the data detection information comprises initial detection information and additional detection information corresponding to the current detection period; detecting the data to be detected according to the initial detection information and the additional detection information, and obtaining data quality information corresponding to the data to be detected; generating a data set to be analyzed based on the detected data to be detected, and carrying out statistical analysis on the data set to be analyzed to obtain statistical analysis information corresponding to the data set to be analyzed; and generating target detection information corresponding to the current detection period based on the data quality information and the statistical analysis information. And carrying out multidimensional detection on the data to be detected through the initial detection information and the dynamically set additional detection information, so as to meet the requirement of customized detection.
Description
Technical Field
The present disclosure relates to the field of data processing technologies, and in particular, to a data detection method. The present application is also directed to a data detection apparatus, a computing device, and a computer-readable storage medium.
Background
With the development of internet technology, the data size of software applications is increasing. During the running of a software application, the real-time state of the running of the software is usually recorded in text form in a file, called a log file, based on a buried point manner, for recording detailed running information during the running of the software application. The data in the log file may be collected later to analyze the usage of the software application, the usage habits of the user, etc. Therefore, in order to analyze the correct result later, it is necessary to ensure that the data access process is not in error, so that the acquired data is reliable, and at present, the data is manually verified and detected abnormally according to past experience by a developer, and the mode of manually checking the data acquisition abnormality consumes a great deal of manpower and has low detection efficiency. Therefore, how to improve the data anomaly detection efficiency is a problem that needs to be solved at present.
Disclosure of Invention
In view of this, the embodiments of the present application provide a data detection method. The application relates to a data detection device, a computing device and a computer readable storage medium, so as to solve the problems of complex data anomaly detection steps and low efficiency in the prior art.
According to a first aspect of an embodiment of the present application, there is provided a data detection method, including:
acquiring data to be detected in a current detection period, and determining data detection information of the data to be detected, wherein the data detection information comprises initial detection information and additional detection information corresponding to the current detection period;
detecting the data to be detected according to the initial detection information and the additional detection information to obtain data quality information corresponding to the data to be detected;
generating a data set to be analyzed based on the detected data to be detected, and carrying out statistical analysis on the data set to be analyzed to obtain statistical analysis information corresponding to the data set to be analyzed;
and generating target detection information corresponding to the current detection period based on the data quality information and the statistical analysis information.
According to a second aspect of embodiments of the present application, there is provided a data detection apparatus, including:
the determining module is configured to acquire data to be detected in a current detection period and determine data detection information of the data to be detected, wherein the data detection information comprises initial detection information and additional detection information corresponding to the current detection period;
The detection module is configured to detect the data to be detected according to the initial detection information and the additional detection information, and obtain data quality information corresponding to the data to be detected;
the analysis module is configured to generate a data set to be analyzed based on the detected data to be detected, and perform statistical analysis on the data set to be analyzed to obtain statistical analysis information corresponding to the data set to be analyzed;
and the generation module is configured to generate target detection information corresponding to the current detection period based on the data quality information and the statistical analysis information.
According to a third aspect of embodiments of the present application, there is provided a computing device comprising a memory, a processor and computer instructions stored on the memory and executable on the processor, the processor implementing the steps of the data detection method when executing the computer instructions.
According to a fourth aspect of embodiments of the present application, there is provided a computer readable storage medium storing computer instructions which, when executed by a processor, implement the steps of the data detection method.
According to the data detection method, data to be detected in a current detection period are obtained, and data detection information of the data to be detected is determined, wherein the data detection information comprises initial detection information and additional detection information corresponding to the current detection period; detecting the data to be detected according to the initial detection information and the additional detection information to obtain data quality information corresponding to the data to be detected; generating a data set to be analyzed based on the detected data to be detected, and carrying out statistical analysis on the data set to be analyzed to obtain statistical analysis information corresponding to the data set to be analyzed; and generating target detection information corresponding to the current detection period based on the data quality information and the statistical analysis information.
According to the embodiment of the application, when the data to be detected is detected, the data to be detected is subjected to multi-dimensional detection through the initial detection information and the additional detection information, the additional detection information can be dynamically set according to actual requirements, and the customized detection requirements are met. The data to be detected is automatically detected, the data set to be analyzed is subjected to statistical analysis, manual detection by an acceptance person is not needed, the detection result of the data to be detected can be quickly obtained based on target detection information, and the data detection efficiency is improved.
Drawings
FIG. 1 is a schematic diagram of a data detection page according to an embodiment of the present application
FIG. 2 is a flow chart of a data detection method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a detection information document of a data detection method according to an embodiment of the present application;
fig. 4 is a schematic diagram of data distribution of a data detection method according to an embodiment of the present application;
FIG. 5 is a process flow diagram of a data detection method for game log detection according to one embodiment of the present application;
FIG. 6 is a schematic diagram of a data detection device according to an embodiment of the present disclosure;
FIG. 7 is a block diagram of a computing device according to one embodiment of the present application.
Detailed Description
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is, however, susceptible of embodiment in many other ways than those herein described and similar generalizations can be made by those skilled in the art without departing from the spirit of the application and the application is therefore not limited to the specific embodiments disclosed below.
The terminology used in one or more embodiments of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments of the application. As used in this application in one or more embodiments and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present application refers to and encompasses any or all possible combinations of one or more of the associated listed items.
It should be understood that, although the terms first, second, etc. may be used in one or more embodiments of the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, a first may also be referred to as a second, and similarly, a second may also be referred to as a first, without departing from the scope of one or more embodiments of the present application. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.
At present, when a game is in a testing stage or an operation stage, game data obtained by a data access person in a buried point mode is detected by a game acceptance person, and whether the game log data meets design requirements is judged. The existing data detection system only detects a part of small data volume to reflect the quality of all data, but the mode easily leaks error data, so that faults cannot be found in time, and potential safety hazards exist. Secondly, the game acceptance personnel are required to manually accept the buried point data during verification, and feedback is carried out to the data access personnel after the problem is found, so that the process efficiency is low, time and labor are consumed, and the access and detection of the data are not facilitated.
Based on the data detection method, the data detection method is provided, and is used for automatically detecting and analyzing the data, so that the data detection efficiency is improved. The present application relates to a data detection apparatus, a computing device, and a computer-readable storage medium, which are described in detail in the following embodiments.
Fig. 1 is a schematic diagram of a data detection page of a data detection method according to an embodiment of the present application, wherein target detection information is presented in the data detection page after the data detection is completed. After the server receives the game log data, the received game log data is detected, data detection is carried out through initial detection information and additional detection information in the embedded point document, the initial detection information comprises basic format requirements of the data, the additional detection information comprises formula detection requirements of the data, and after the data to be detected are detected according to the initial detection information and the additional detection information, the data quality information of the current data to be detected can be obtained. The data to be detected includes data of a plurality of different message types, so that quality information of the data of each message type can be obtained. As in fig. 1, the data quality information of the connection server message type data may include the total number of data, the detected number, the number of errors, whether the detection is passed, the cause of the errors, and the like. The data quality information is fed back into the data detection page, so that detection details can be clearly displayed for data acceptance personnel, and the detected data can be subjected to statistical analysis later, so that data validation personnel can observe whether the data meets design requirements or not. According to the data processing method, the data to be detected can be detected in various different verification modes through the initial detection information and the additional detection information, statistical analysis can be further carried out on the detected data to be detected before, statistical analysis information is obtained, target detection information is generated based on the data quality information and the statistical analysis information, and the target detection information is fed back to the data acceptance person based on the data detection page, so that the data acceptance person can rapidly accept game log data without manually accepting the game log data, and the data detection efficiency is improved. In the detection process, data acceptance personnel can customize additional detection information according to acceptance standards, and the data to be detected is dynamically detected, so that the data detection method accords with data detection in various different scenes. The data after detection can be subjected to statistical analysis later, so that game developers can observe whether the data meets the design requirements, and daily maintenance or repair of the game is facilitated.
Fig. 2 shows a flowchart of a data detection method according to an embodiment of the present application, which specifically includes the following steps:
step 202: and acquiring data to be detected in a current detection period, and determining data detection information of the data to be detected, wherein the data detection information comprises initial detection information and additional detection information corresponding to the current detection period.
The current detection period can be understood as a period of the current detection data, and because the server receives log data in real time, when a data inspector prepares to accept the data, the current received log data which is not detected yet starts to be detected based on the client, namely, the data detection in the current detection period starts. The data to be detected can be understood as object data to be detected in the current detection period, the data to be detected can be software operation log data, buried point data collected by data buried point personnel, and even some daily data such as resume collected in daily life. The initial detection information can be a preset data basic format, whether the data format of the data to be detected meets the preset requirement can be judged through the initial detection information, if the format requirement of the data of the preset A type is character type, if the data of the floating point type exists in the data of the A type, the data can be determined to be error data when the data is detected. The additional detection information may be understood as a detection requirement set by the data acceptance person in a self-defining manner, if the additional detection information may be "app id= =1102", whether the value of the app id type data is equal to 1102 is determined according to the additional detection information, and if not, the data is determined to be error data. The data acceptance personnel can manually set the additional detection information, so that the data to be detected is detected under different detection periods, and the additional detection information can be dynamically adjusted according to actual detection requirements, so that the data detection under various different detection scenes is satisfied.
In practical application, the initial detection information and the additional detection information may be stored in a detection information document, and the specific detection information text may be a buried data document set by an inspector, as shown in fig. 3, fig. 3 is a schematic diagram of the detection information document in the data detection method provided in an embodiment of the present application, where the detection information document includes two types of initial detection information and additional detection information, where the initial detection information includes a parameter type, a parameter length, and the like, and is used to determine whether parameter data meets a required parameter type, length, and the like, and the additional detection information is a detection formula manually set by the inspector for data, and determines whether the data meets a set requirement.
In conclusion, the data to be detected is detected through the additional detection information, customized detection of the data is achieved, different detection scenes are met, manual detection by data acceptance personnel is not needed, and detection efficiency is improved.
Step 204: and detecting the data to be detected according to the initial detection information and the additional detection information, and obtaining data quality information corresponding to the data to be detected.
After the initial detection information and the additional detection information are determined, the data to be detected can be detected based on the initial detection information and the additional detection information, after the detection is completed, the data quality information corresponding to the data to be detected can be obtained, the data quality information can be understood as the detection result of the data to be detected, the data quality information can comprise the detection total number, the detected number, the error reasons and the like of the data to be detected, the data quality of the data to be detected can be quickly and clearly known by a data acceptance person through the data quality information, and the data can be quickly fed back to a game developer under the condition that the data is in error detection, so that the game developer is reminded of fault repair, and unnecessary loss is avoided.
In practical application, because the detection modes of the data of different data types are different, in order to detect all types of data and prevent the condition of missed detection, corresponding additional detection information can be set for different data types, and the data to be detected is detected through the additional detection information, and specifically, the method comprises the following steps: performing format detection on the data to be detected based on the initial detection information; performing formula detection on the data to be detected based on the additional detection information; and determining the data quality information corresponding to the data to be detected according to the format detection result and the formula detection result.
The format detection may be understood as detecting a data format of the data to be detected according to a format requirement preset in the initial detection information, for example, in fig. 3, setting a parameter type of msgid type parameter data as string, and if some piece of data of msgid type in the acquired data to be detected is not string type, indicating that the data is in error. The formula detection can be understood as detecting the data to be detected according to a detection formula set in the additional detection information, if the additional detection information of the app id type is set to be "app id= =1102", if the parameter data of the app id type in the data to be detected is not 1102, it is indicated that the parameter does not meet the detection requirement, and there is an error.
In specific implementation, the accessory detection information can be a detection formula set by a data acceptance person, the formula indicates the dynamic detection of the and/or multiple conditions, and the multiple fields are matched for detection, so that a more comprehensive detection mode is provided, and various different detection requirements are met. After format detection is performed on the data to be detected based on the initial detection information, a format detection result of the data to be detected can be obtained, after formula detection is performed on the data to be detected based on the additional detection information, a formula detection result of the data to be detected can be obtained, data quality information corresponding to the data to be detected can be determined according to the format detection result and the formula detection result, and when the format detection result and the formula detection result are not in error, the data quality to be detected is high, and error data are not present.
In a specific embodiment of the present application, initial detection information for determining data of an app id type in data to be detected is "data type: string, data length: 40", detecting the app id type data in the data to be detected based on the initial detection information, if the data length of a certain app id type data is 45, indicating that the app id type data is in error, and recording the app id type data and the corresponding error reason in the format detection result. Determining that the additional detection information is 'app id= 100', performing formula detection on app id type data in the data to be detected based on the additional detection information, if the value of a certain piece of app id type data is 101, indicating that the piece of app id type data is in error, and recording the piece of app id type data and a corresponding error reason in a formula detection result.
In another specific embodiment of the present application, the initial detection information for determining the msgVersion message type in the data to be detected is "data type: double, data length: and 40", detecting the data of the msgVersion message type in the data to be detected based on the initial detection information and storing the format detection result. Determining that the additional detection information is "(data source= = 'client' andmsgversion= =2.0) or (data source= = 'server' andmsgversion= =2.1)", performing formula detection on msgVersion message type data in the data to be detected based on the additional detection information, and if the data source of a certain piece of data is a client, judging whether the version number of the client is 2.0 according to a detection formula; if the data source of a certain piece of data is a server, judging whether the version number of the server is 2.1 according to a detection formula, and storing a formula detection record after formula detection is completed. Dynamic detection is performed through formula detection containing various detection conditions, so that cooperation detection among multiple fields can be realized, a comprehensive detection method is provided, and various different detection scenes are satisfied.
Based on the data quality information of the data to be detected is generated based on the format detection result and the formula detection result after the detection is completed, and is subsequently used for being fed back to the data acceptance personnel, so that the data acceptance personnel can quickly determine error data and error reasons, and the subsequent data restoration efficiency is improved.
In practical application, when error data exists in data to be detected, a data acceptance person is prevented from manually traversing the data to be detected to find the error data, so that the detected error data can be directly recorded and fed back to the data acceptance person after detection is completed, and the data quality information corresponding to the data to be detected is specifically determined according to a format detection result and a formula detection result, and the method comprises the following steps: determining problem information corresponding to problem sub-data in the data to be detected according to a format detection result and a formula detection result; and generating data quality information corresponding to the data to be detected based on the problem sub-data and the problem information.
The problem sub-data may be understood as data with errors, the problem sub-data may be format detection errors or formula detection errors, if one of format detection or formula detection errors exists, the piece of data may be determined to be problem sub-data, the problem information may be understood as specific error reasons of the problem sub-data, and if the data length exceeds the set maximum length, the problem information of the problem sub-data may be "length: wrong ".
Based on the above, when error data exists in the data to be detected, the data quantity and the error reasons of the error detection can be recorded and displayed to the data acceptance personnel. The data acceptance person can click the error reason to specifically check the detailed data of the data, so that a mode of quickly determining the error data is provided for the data acceptance person, and unnecessary resource waste is caused because the data acceptance person also needs to manually search the error data after determining that the data to be detected is in error.
Further, since there may be multiple pieces of error data in the data to be detected, when the data acceptance person determines the problem data, the data acceptance person is prevented from missing the problem data, so that the problem data is not repaired subsequently, and therefore, the data quantity of the problem data can be explicitly recorded in the data quality information, specifically, the data quality information corresponding to the data to be detected is generated based on the problem sub-data and the problem information, including: determining a data entry corresponding to the problem sub-data; and generating data quality information corresponding to the data to be detected according to the data entry and the problem information.
The data entries may be understood as the number of problem sub-data, if there are 100 problem sub-data in the data to be detected, the data entries of the problem sub-data are 100, and the following data quality information may explicitly indicate that there are 100 problem sub-data.
In summary, the data acceptance personnel can know the number of the problem sub-data through the number of the problem sub-data in the data quality information, so that the condition of missing the problem sub-data is avoided. The game developer can repair the game according to the problem sub-data conveniently, and better game experience is provided for the player.
Further, because the data size of the data to be detected is large, the time possibly spent in detecting the data to be detected is more, so that the data acceptance personnel does not know the detection progress, and therefore, the detection progress bar can be fed back to the data acceptance personnel when the data is detected, and the method specifically comprises the following steps: determining a detection progress control corresponding to the data to be detected; determining detected data in data to be detected in a detection state, and the data duty ratio of the detected data in the data to be detected; and updating the detection progress control according to the data duty ratio, and displaying the updated detection progress control.
The detection progress control can be understood as a control for presenting detection progress, and the control can be in a form of a progress bar or a form of a progress number. In the process of detecting the data to be detected, the detected data can be determined, namely the detected data is the data which is detected, the data proportion of the detected data in the data to be detected is determined, if the data to be detected is 100 pieces, and the detected data is 50 pieces, the determined data proportion is 0.5, the detection progress control can be updated according to the data proportion, and when the data proportion is 0.5 in the form of a progress bar in the display of fig. 1, the detection progress is 50%, so that a data acceptance person can clearly know the current detection progress.
In practical application, the detection progress control can be updated according to different progress granularities, for example, the detection progress can be updated once according to each piece of data detected, or the detection progress can be updated once according to each piece of data detected, which specifically includes: determining a data entry corresponding to the detected data; and calculating the data duty ratio based on the data item corresponding to the detected data and the data item corresponding to the data to be detected, and obtaining the updated detection progress control.
The data entries corresponding to the detected data can be understood as the data quantity of the detected data, and after the data entries are determined, the data duty ratio can be calculated according to the data entries of the detected data and the data entries of the data to be detected, and the detection progress is updated through the data duty ratio.
In a specific embodiment of the present application, it is determined that a data entry of detected data is 100, a data entry corresponding to data to be detected is 1000, then detection progress information is calculated based on the data entry corresponding to the detected data and the data entry corresponding to the data to be detected, that is, a data duty ratio is 0.1, and then the detection progress control is updated based on the data duty ratio.
In another implementation manner, the progress may be updated according to the data of one data type, that is, after the detection is completed on the data of the type, the detection progress may be updated. The method specifically comprises the following steps: determining an initial data type set corresponding to data to be detected and a target data type set corresponding to the detected data; and under the condition that at least one initial data type is included in the target data type set, calculating a data duty ratio based on the initial data type set and the target data type set, and obtaining an updated detection progress control.
The initial data type set may be understood as all types included in the data to be detected, for example, the initial data type set includes "a type, B type, C type, D type", and the target data type set of the detected data is determined to be "a type", then the data duty ratio is calculated based on the initial data type set and the target data type set to be 0.25, and then the detection progress control is updated according to the data duty ratio.
In conclusion, the data detection progress can be determined according to different actual requirements by updating the data detection progress at different granularities, so that data acceptance personnel can obtain the data detection progress more accurately.
Step 206: and generating a data set to be analyzed based on the detected data to be detected, and carrying out statistical analysis on the data set to be analyzed to obtain statistical analysis information corresponding to the data set to be analyzed.
The data set to be analyzed can be understood as a data set formed by detected data to be detected, after the data to be detected is detected, statistical analysis can be performed on all detected data to be detected, statistical analysis can be understood as whether distribution of each type of data in the data to be detected meets design requirements or not, statistical analysis information can be understood as data distribution information of the data set to be analyzed, the statistical analysis information can comprise a data distribution map, a data analysis result and the like, and game developers can clearly know the distribution situation of current data according to the statistical analysis information, so that game setting is adjusted according to the distribution situation, and game development meets the design requirements.
In practical applications, taking player level type data in data to be detected as an example, after statistical analysis is performed on a player level data set, a corresponding data distribution diagram may be obtained, as shown in fig. 4, fig. 4 is a schematic data distribution diagram of a data detection method provided in an embodiment of the present application, and in fig. 4, the number of persons at each level is presented, where the level of each player may be observed. If the number of people at a certain level is abnormal, the task difficulty set at the level in the game is possibly unreasonable, and game developers can adjust the game content at the level, so that the player levels are uniformly distributed and meet the game design requirement.
Further, in order to accurately observe the distribution condition of the data to be analyzed and avoid errors in statistical analysis, the analysis can be performed based on the quartile statistics, and specifically includes: determining the quantile statistical information corresponding to the data set to be analyzed based on the data to be analyzed in the data set to be analyzed; generating data distribution information corresponding to the data set to be analyzed based on the bit statistical information and the data set to be analyzed, and taking the data distribution information as statistical analysis information corresponding to the data set to be analyzed.
The term "statistical information" is understood to mean the quartile calculated from the data to be analyzed. The quartile statistics may be understood as dividing a group of statistical data into four equal divisions in order from small to large, and dividing the data division points of the four divisions into quartiles, and when the method is implemented, the total of the quartiles is three: a first quartile, the value below the first quartile being the lowest 25% of the sample data; a second quartile, the second quartile being the median of the entire set of data; the third quartile, the value higher than the third quartile, is the highest 25% of the sample data; greater than the third quartile is the highest 25% fraction.
In practical application, after determining the quartile corresponding to the data to be analyzed, corresponding data distribution information can be generated according to the data to be analyzed and the data set to be analyzed, wherein the data distribution information can be understood as data distribution diagram or information generated according to the data distribution diagram or information according to the quartile distribution to be analyzed, and the data distribution information is taken as the statistical distribution information.
Based on the statistical distribution information, a game developer can observe the data distribution condition based on the data to be analyzed, so that the game developer can quickly acquire the data distribution condition, adjust the game in time and guarantee the game experience of a player.
Furthermore, since the data acceptance personnel cannot determine the data distribution condition according to the data to be detected, in order to obtain the data distribution condition more clearly, the data distribution can be performed on the data to be analyzed according to the statistical information, which specifically includes: classifying the data of the data set to be analyzed according to the bit statistical information, and determining at least two sub-data sets to be analyzed according to the classification result; and respectively carrying out data sequencing on each sub-data set to be analyzed, and determining data distribution information corresponding to the data set to be analyzed according to sequencing results corresponding to each sub-data set to be analyzed.
The data classification is performed on the data set to be analyzed, which can be understood as equally dividing the data to be analyzed according to quartiles, namely dividing the data to be analyzed into four equal parts, and the following table is a result of performing data classification on the data of the diagonal color grade rollevel type provided in an embodiment of the present application, specifically see table 1:
TABLE 1
| Field name | Field description | Index (I) | Recording the value | |
| rolerolevel | Role grade | Min | 1 | |
| rolerolevel | Role grade | 1/4 |
15 | |
| rolerolevel | Role grade | Median of | 40 | |
| | Role grade | 3/4 quantiles | 55 | |
| rolerolevel | Role grade | Max | 120 | |
| rolerolevel | Role grade | Mode number | 62 |
In table 1, the index can be understood as the quantile statistical information which needs to be calculated according to the data to be analyzed, including 1/4 quantile, median and 3/4 quantile, and in order to better perform statistics on the data to be analyzed, a minimum value Min, a maximum value Max and a mode can be added. In the table, the data classification can be defined for the rolerolevel message types with the color classes of 1 to 120, the calculated quantile statistical information is 15 for 1/4 quantile points, namely 40 for 1/2 quantile points, and 55 for 3/4 quantile points, and the data of the rolerolevel type can be equally divided according to the calculated quantile statistical information to obtain four equally divided sub-data sets to be analyzed. The data to be analyzed in each equal division is combined into a sub-data set to be analyzed, the sub-data to be analyzed in the sub-data set to be analyzed is subjected to data sorting, and sorting is performed according to the order from small to large, so that the data quantity on each numerical value can be determined, as in fig. 4, the player quantity on each player level is determined, and after the number of the sub-data to be analyzed on each numerical value is determined, the data distribution information corresponding to the data set to be analyzed can be determined.
In a specific embodiment of the present application, the data set to be analyzed is a player level data set, after the player level data is divided into four parts according to the division statistics information, four player data sub-data sets can be obtained, each player level sub-data set is subjected to data sorting, and the number of data on each player level is determined, so that the number of players on each player level can be obtained, and data distribution information corresponding to the player data sets, namely, a data distribution diagram, can be generated according to the number of players on each player level. The subsequent game developer can obtain the distribution condition of the current player level according to the data distribution diagram, determine whether the current player level distribution meets the design requirement, and adjust the distribution condition if not, so that the game developer can maintain the game environment conveniently, and the game experience of the player is improved.
Step 208: and generating target detection information corresponding to the current detection period based on the data quality information and the statistical analysis information.
The target detection information can be understood as detection result information displayed to the data acceptance person, after the data quality information and the statistical analysis information are determined, the data quality information and the statistical analysis information can be combined to generate target detection information of the detected data to be detected, and the target detection information can be displayed to the data acceptance person subsequently, so that the data acceptance person can observe the detailed content of the detection through the target detection information.
In practical application, in order to facilitate the data acceptance personnel to clearly understand the detection result, the target detection information may be displayed in the data detection page, which specifically includes: adding the target detection information to a data detection page and displaying; receiving a data restoration instruction returned by a target user based on the data detection page; and determining data restoration information in response to the data restoration instruction, and restoring the data to be detected based on the data restoration information.
The data detection page may be understood as a page showing a detection result to a data acceptance person after the data detection system detects the data to be detected, where the data detection page is shown in fig. 1, and includes a data message type, a message format, the number of data of each message type and the detected number in the data to be detected.
In practical application, when the target detection information includes error data, the error data and the corresponding error reasons thereof can be displayed in the data detection page, after the data acceptance personnel determine that the current data to be detected has errors, the error reasons can be reported, and the game developer or the data burial point personnel issue a data repair instruction which can be understood as an instruction for repairing the error data. If the error is caused by the error of the burying point setting of the data burying point personnel, the data repairing instruction can be used for adjusting the burying point setting, so that the correctness of the data collected by the subsequent burying point is ensured. Data repair information may be understood as methods, steps for error data repair.
In summary, the running software corresponding to the data to be detected can be repaired through the data repair information, so that unnecessary resource loss is avoided.
Further, in order to prevent the situation that the data repair fails but the data acceptance personnel does not find, the secondary detection can be performed on the repaired data, which specifically includes: determining a data screening condition, and determining target detection data in the repaired data to be detected according to the data screening condition; and detecting the target detection data according to the initial detection information and the additional detection information to obtain target data quality information corresponding to the target detection data.
The data screening conditions can be understood as conditions for screening target detection data, the data screening conditions can include time screening, data type screening and the like, and data to be detected received after the data inspector wants to detect four points can be screened out according to the data screening conditions; or only want to detect the data of a certain data type, can set up the screening condition as this type screening condition, can confirm the goal detection data belonging to this data type according to the screening condition of the data.
Based on the method, the target detection data is subjected to secondary detection through the initial detection information and the additional detection information, and the target data quality information of the target detection data is determined, so that whether the data restoration is effective is determined, and the problem of abnormal game operation caused by failure of data restoration is avoided.
The data detection method comprises the steps of obtaining data to be detected in a current detection period, and determining data detection information of the data to be detected, wherein the data detection information comprises initial detection information and additional detection information corresponding to the current detection period; detecting the data to be detected according to the initial detection information and the additional detection information to obtain data quality information corresponding to the data to be detected; generating a data set to be analyzed based on the detected data to be detected, and carrying out statistical analysis on the data set to be analyzed to obtain statistical analysis information corresponding to the data set to be analyzed; and generating target detection information corresponding to the current detection period based on the data quality information and the statistical analysis information. The multi-dimensional detection of the data to be detected is realized through the initial detection information and the additional detection information when the data to be detected is detected, and the additional detection information can be dynamically set according to the actual requirements, so that the customized detection requirements are met. The data to be detected is automatically detected, the data set to be analyzed is subjected to statistical analysis, manual detection by an acceptance person is not needed, the detection result of the data to be detected can be quickly obtained based on target detection information, and the data detection efficiency is improved.
The application of the data detection method provided in the present application to game log detection is taken as an example, and the data detection method is further described below with reference to fig. 5. Fig. 5 shows a process flow chart of a data detection method applied to game log detection according to an embodiment of the present application, which specifically includes the following steps:
step 502: and acquiring data to be detected in the current detection period, and determining initial detection information of the data to be detected and additional detection information corresponding to the current detection period.
In one implementation, the data to be detected is game log data, initial detection information and additional detection information of the game log data are determined, the initial detection information is basic format detection information in the embedded point document, and the additional detection information is dynamic formula detection information in the embedded point document.
Step 504: and carrying out format detection on the data to be detected based on the initial detection information, and carrying out formula detection on the data to be detected based on the additional detection information.
In one implementation, the game log data is format detected based on the format detection information, and the game log data is formula detected based on the dynamic formula detection information.
Step 506: and determining problem information corresponding to the problem sub-data in the data to be detected according to the format detection result and the formula detection result, and generating data quality information corresponding to the data to be detected based on the problem sub-data and the problem information.
In one implementation manner, according to the results obtained after the detection in two different detection manners, the problem sub-data of the detection failure in the data to be detected and the problem information corresponding to the problem sub-data are determined, and the problem information is the problem reason. And generating data quality information of the game log data, namely a data detection result, according to the problem sub-data and the problem information. The number of the problem sub-data failing to be detected can also be added into the data quality information, and the data quality information is displayed through the data detection page.
In the detection process, the current detection progress can be displayed, including determining a detection progress control corresponding to the data to be detected; determining detected data in data to be detected in a detection state, and the data duty ratio of the detected data in the data to be detected; and updating the detection progress control according to the data duty ratio, and displaying the updated detection progress control.
In another implementation manner, a detection progress bar of the current detection data is determined, the detected data which is already detected currently is determined, the data proportion of the detected data in all data to be detected is determined, and the detection progress bar is updated and displayed according to the data proportion.
Step 508: generating a data set to be analyzed based on the detected data to be detected, and determining the quantile statistical information corresponding to the data set to be analyzed based on the data to be analyzed in the data set to be analyzed.
In one implementation, a data set to be analyzed is generated according to the detected game log data, and the data set to be analyzed is subjected to quartile statistical analysis to determine the corresponding quartile of the data set to be analyzed.
Step 510: and classifying the data of the data set to be analyzed according to the bit statistical information, and determining four sub-data sets to be analyzed according to the classification result.
In one implementation, data in the data set to be analyzed is categorized according to quartiles, and the data to be analyzed is equally divided into four parts, so that four sub-data sets to be analyzed are determined.
Step 512: and respectively carrying out data sequencing on each sub-data set to be analyzed, and determining data distribution information corresponding to the data set to be analyzed according to sequencing results corresponding to each sub-data set to be analyzed.
In one implementation manner, the data of each sub-data set to be analyzed is respectively sequenced, and the number of the sub-data to be analyzed on each numerical value is determined according to the sequencing result, so that data distribution information corresponding to the data set to be analyzed is generated.
Step 514: and generating target detection information corresponding to the current detection period based on the data quality information and the statistical analysis information.
In one implementation, the target detection information in the current detection period is generated according to a combination of the data quality information and the statistical analysis information.
Step 516: and adding the target detection information to a data detection page, displaying the target detection information, receiving a data restoration instruction returned by a target user based on the data detection page, responding to the data restoration instruction to determine data restoration information, and restoring the data to be detected based on the data restoration information.
In one implementation manner, the target detection information is added to the data detection page and is displayed to the data acceptance person, the data repair instruction returned by the data acceptance person based on the data detection page is received, and the data to be detected is repaired based on the data repair instruction.
Step 518: and determining data screening conditions, and determining target detection data in the repaired data to be detected according to the data screening conditions.
In one implementation, target detection data for data repair is quickly determined based on data screening conditions.
Step 520: and detecting the target detection data according to the initial detection information and the additional detection information to obtain target data quality information corresponding to the target detection data.
In one implementation, the target detection data is secondarily detected according to the initial detection information and the additional detection information, and a data detection result of the secondary detection is obtained.
According to the data detection method, when the data to be detected is detected, the data to be detected is subjected to multi-dimensional detection through the initial detection information and the additional detection information, the additional detection information can be dynamically set according to actual requirements, and the customized detection requirements are met. The detection progress can be displayed in the detection process, so that the data acceptance personnel can clearly know the current detection progress, follow-up automatic detection is carried out on the data to be detected and statistical analysis is carried out on the data set to be analyzed, the acceptance personnel do not need to manually detect, the detection result of the data to be detected can be quickly obtained based on the target detection information, and the data detection efficiency is improved.
Corresponding to the method embodiment, the present application further provides an embodiment of a data detection device, and fig. 6 shows a schematic structural diagram of the data detection device according to an embodiment of the present application. As shown in fig. 6, the apparatus includes:
a determining module 602, configured to obtain data to be detected in a current detection period, and determine data detection information of the data to be detected, where the data detection information includes initial detection information and additional detection information corresponding to the current detection period;
the detection module 604 is configured to detect the data to be detected according to the initial detection information and the additional detection information, and obtain data quality information corresponding to the data to be detected;
the analysis module 606 is configured to generate a data set to be analyzed based on the detected data to be detected, and perform statistical analysis on the data set to be analyzed to obtain statistical analysis information corresponding to the data set to be analyzed;
a generating module 608 is configured to generate target detection information corresponding to the current detection period based on the data quality information and the statistical analysis information.
Optionally, the detecting module 604 is further configured to: performing format detection on the data to be detected based on the initial detection information; performing formula detection on the data to be detected based on the additional detection information; and determining the data quality information corresponding to the data to be detected according to the format detection result and the formula detection result.
Optionally, the detecting module 604 is further configured to: determining problem information corresponding to problem sub-data in the data to be detected according to a format detection result and a formula detection result; and generating data quality information corresponding to the data to be detected based on the problem sub-data and the problem information.
Optionally, the detecting module 604 is further configured to: determining a data entry corresponding to the problem sub-data; and generating data quality information corresponding to the data to be detected according to the data entry and the problem information.
Optionally, the apparatus further includes a progress module configured to: determining a detection progress control corresponding to the data to be detected; determining detected data in data to be detected in a detection state, and the data duty ratio of the detected data in the data to be detected; and updating the detection progress control according to the data duty ratio, and displaying the updated detection progress control.
Optionally, the analysis module 606 is further configured to: determining the quantile statistical information corresponding to the data set to be analyzed based on the data to be analyzed in the data set to be analyzed; generating data distribution information corresponding to the data set to be analyzed based on the bit statistical information and the data set to be analyzed, and taking the data distribution information as statistical analysis information corresponding to the data set to be analyzed.
Optionally, the analysis module 606 is further configured to: classifying the data of the data set to be analyzed according to the bit statistical information, and determining at least two sub-data sets to be analyzed according to the classification result; and respectively carrying out data sequencing on each sub-data set to be analyzed, and determining data distribution information corresponding to the data set to be analyzed according to sequencing results corresponding to each sub-data set to be analyzed.
Optionally, the analysis module 606 is further configured to: adding the target detection information to a data detection page and displaying; receiving a data restoration instruction returned by a target user based on the data detection page; and determining data restoration information in response to the data restoration instruction, and restoring the data to be detected based on the data restoration information.
Optionally, the analysis module 606 is further configured to: determining a data screening condition, and determining target detection data in the repaired data to be detected according to the data screening condition; and detecting the target detection data according to the initial detection information and the additional detection information to obtain target data quality information corresponding to the target detection data.
The application provides a data detection device, include: the determining module is configured to acquire data to be detected in a current detection period and determine data detection information of the data to be detected, wherein the data detection information comprises initial detection information and additional detection information corresponding to the current detection period; the detection module is configured to detect the data to be detected according to the initial detection information and the additional detection information, and obtain data quality information corresponding to the data to be detected; the analysis module is configured to generate a data set to be analyzed based on the detected data to be detected, and perform statistical analysis on the data set to be analyzed to obtain statistical analysis information corresponding to the data set to be analyzed; and the generation module is configured to generate target detection information corresponding to the current detection period based on the data quality information and the statistical analysis information. When the data to be detected is detected, the data to be detected is subjected to multidimensional detection through the initial detection information and the additional detection information, and the additional detection information can be dynamically set according to actual requirements, so that the customized detection requirements are met. The data to be detected is automatically detected, the data set to be analyzed is subjected to statistical analysis, manual detection by an acceptance person is not needed, the detection result of the data to be detected can be quickly obtained based on target detection information, and the data detection efficiency is improved.
The above is a schematic solution of a data detection device of the present embodiment. It should be noted that, the technical solution of the data detection device and the technical solution of the data detection method belong to the same conception, and details of the technical solution of the data detection device, which are not described in detail, can be referred to the description of the technical solution of the data detection method.
Fig. 7 illustrates a block diagram of a computing device 700 provided in accordance with an embodiment of the present application. The components of computing device 700 include, but are not limited to, memory 710 and processor 720. Processor 720 is coupled to memory 710 via bus 730, and database 750 is used to store data.
Computing device 700 also includes access device 740, access device 740 enabling computing device 700 to communicate via one or more networks 760. Examples of such networks include the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. The access device 740 may include one or more of any type of network interface, wired or wireless (e.g., a Network Interface Card (NIC)), such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, a worldwide interoperability for microwave access (Wi-MAX) interface, an ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a bluetooth interface, a Near Field Communication (NFC) interface, and so forth.
In one embodiment of the present application, the above-described components of computing device 700, as well as other components not shown in FIG. 7, may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device illustrated in FIG. 7 is for exemplary purposes only and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as desired.
Computing device 700 may be any type of stationary or mobile computing device including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), mobile phone (e.g., smart phone), wearable computing device (e.g., smart watch, smart glasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 700 may also be a mobile or stationary server.
Wherein the processor 720 performs the steps of the data detection method when executing the computer instructions.
The foregoing is a schematic illustration of a computing device of this embodiment. It should be noted that, the technical solution of the computing device and the technical solution of the data detection method belong to the same concept, and details of the technical solution of the computing device, which are not described in detail, can be referred to the description of the technical solution of the data detection method.
An embodiment of the present application also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the data detection method as described above.
The above is an exemplary version of a computer-readable storage medium of the present embodiment. It should be noted that, the technical solution of the storage medium and the technical solution of the data detection method belong to the same concept, and details of the technical solution of the storage medium which are not described in detail can be referred to the description of the technical solution of the data detection method.
The foregoing describes specific embodiments of the present application. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
The computer instructions include computer program code that may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a Read-only memory (ROM), a random access memory (RAM, randomAccessMemory), an electrical carrier signal, a telecommunication signal, a software distribution medium, and so forth. It should be noted that the computer readable medium contains content that can be appropriately scaled according to the requirements of jurisdictions in which such content is subject to legislation and patent practice, such as in certain jurisdictions in which such content is subject to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.
It should be noted that, for the sake of simplicity of description, the foregoing method embodiments are all expressed as a series of combinations of actions, but it should be understood by those skilled in the art that the present application is not limited by the order of actions described, as some steps may be performed in other order or simultaneously in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily all necessary for the present application.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.
The above-disclosed preferred embodiments of the present application are provided only as an aid to the elucidation of the present application. Alternative embodiments are not intended to be exhaustive or to limit the invention to the precise form disclosed. Obviously, many modifications and variations are possible in light of the teaching of this application. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best understand and utilize the invention. This application is to be limited only by the claims and the full scope and equivalents thereof.
Claims (12)
1. A data detection method, comprising:
acquiring data to be detected in a current detection period, and determining data detection information of the data to be detected, wherein the data detection information comprises initial detection information and additional detection information corresponding to the current detection period;
detecting the data to be detected according to the initial detection information and the additional detection information to obtain data quality information corresponding to the data to be detected;
generating a data set to be analyzed based on the detected data to be detected, and carrying out statistical analysis on the data set to be analyzed to obtain statistical analysis information corresponding to the data set to be analyzed;
and generating target detection information corresponding to the current detection period based on the data quality information and the statistical analysis information.
2. The method of claim 1, wherein detecting the data to be detected according to the initial detection information and the additional detection information to obtain data quality information corresponding to the data to be detected, comprises:
performing format detection on the data to be detected based on the initial detection information;
performing formula detection on the data to be detected based on the additional detection information;
And determining the data quality information corresponding to the data to be detected according to the format detection result and the formula detection result.
3. The method of claim 2, wherein determining the data quality information corresponding to the data to be detected based on the format detection result and the formula detection result comprises:
determining problem information corresponding to problem sub-data in the data to be detected according to a format detection result and a formula detection result;
and generating data quality information corresponding to the data to be detected based on the problem sub-data and the problem information.
4. The method of claim 3, wherein generating data quality information corresponding to the data to be detected based on the problem sub-data and the problem information comprises:
determining a data entry corresponding to the problem sub-data;
and generating data quality information corresponding to the data to be detected according to the data entry and the problem information.
5. The method of claim 1, wherein the method further comprises:
determining a detection progress control corresponding to the data to be detected;
determining detected data in data to be detected in a detection state, and the data duty ratio of the detected data in the data to be detected;
And updating the detection progress control according to the data duty ratio, and displaying the updated detection progress control.
6. The method of claim 1, wherein performing statistical analysis on the data set to be analyzed to obtain statistical analysis information corresponding to the data set to be analyzed, comprises:
determining the quantile statistical information corresponding to the data set to be analyzed based on the data to be analyzed in the data set to be analyzed;
generating data distribution information corresponding to the data set to be analyzed based on the bit statistical information and the data set to be analyzed, and taking the data distribution information as statistical analysis information corresponding to the data set to be analyzed.
7. The method of claim 6, wherein generating data distribution information corresponding to the set of data to be analyzed based on the quantile statistics and the set of data to be analyzed comprises:
classifying the data of the data set to be analyzed according to the bit statistical information, and determining at least two sub-data sets to be analyzed according to the classification result;
and respectively carrying out data sequencing on each sub-data set to be analyzed, and determining data distribution information corresponding to the data set to be analyzed according to sequencing results corresponding to each sub-data set to be analyzed.
8. The method of claim 1, further comprising, after generating target detection information corresponding to the current detection period based on the data quality information and the statistical analysis information:
adding the target detection information to a data detection page and displaying;
receiving a data restoration instruction returned by a target user based on the data detection page;
and determining data restoration information in response to the data restoration instruction, and restoring the data to be detected based on the data restoration information.
9. The method of claim 8, wherein the method further comprises:
determining a data screening condition, and determining target detection data in the repaired data to be detected according to the data screening condition;
and detecting the target detection data according to the initial detection information and the additional detection information to obtain target data quality information corresponding to the target detection data.
10. A data detection apparatus, comprising:
the determining module is configured to acquire data to be detected in a current detection period and determine data detection information of the data to be detected, wherein the data detection information comprises initial detection information and additional detection information corresponding to the current detection period;
The detection module is configured to detect the data to be detected according to the initial detection information and the additional detection information, and obtain data quality information corresponding to the data to be detected;
the analysis module is configured to generate a data set to be analyzed based on the detected data to be detected, and perform statistical analysis on the data set to be analyzed to obtain statistical analysis information corresponding to the data set to be analyzed;
and the generation module is configured to generate target detection information corresponding to the current detection period based on the data quality information and the statistical analysis information.
11. A computing device comprising a memory, a processor, and computer instructions stored on the memory and executable on the processor, wherein the processor, when executing the computer instructions, performs the steps of the method of any one of claims 1-9.
12. A computer readable storage medium storing computer instructions which, when executed by a processor, implement the steps of the method of any one of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310007562.8A CN115981964A (en) | 2023-01-04 | 2023-01-04 | Data detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310007562.8A CN115981964A (en) | 2023-01-04 | 2023-01-04 | Data detection method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115981964A true CN115981964A (en) | 2023-04-18 |
Family
ID=85973941
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310007562.8A Pending CN115981964A (en) | 2023-01-04 | 2023-01-04 | Data detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115981964A (en) |
-
2023
- 2023-01-04 CN CN202310007562.8A patent/CN115981964A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106294120B (en) | Method, apparatus and computer program product for testing code | |
| CN112990870B (en) | Inspection file generation method and device based on nuclear power equipment and computer equipment | |
| CN110502445B (en) | Software fault severity level judging method and device and model training method and device | |
| CN111400505A (en) | Method and system for matching fault elimination scheme of power consumption information acquisition system | |
| CN109324956B (en) | System testing method, apparatus and computer readable storage medium | |
| CN110941553A (en) | Code detection method, device, equipment and readable storage medium | |
| CN111045902A (en) | Pressure testing method and device for server | |
| CN116414815A (en) | Data quality detection method, device, computer equipment and storage medium | |
| CN112905435B (en) | Workload assessment method, device, equipment and storage medium based on big data | |
| CN113342692B (en) | Automatic test case generation method and device, electronic equipment and storage medium | |
| CN107908525B (en) | Alarm processing method, equipment and readable storage medium | |
| CN111367782A (en) | Method and device for automatically generating regression test data | |
| CN112365269A (en) | Risk detection method, apparatus, device and storage medium | |
| CN116610967A (en) | Bank system abnormality detection method, device and equipment based on clustering | |
| CN115981964A (en) | Data detection method and device | |
| CN107357703B (en) | Terminal application power consumption detection method and server | |
| CN115994093A (en) | Test case recommendation method and device | |
| CN115271277A (en) | Power equipment portrait construction method and system, computer equipment and storage medium | |
| CN115293735A (en) | Unmanned factory industrial internet platform monitoring management method and system | |
| CN115859065A (en) | Model evaluation method, device, equipment and storage medium | |
| CN108235324B (en) | Short message template testing method and server | |
| CN114693116A (en) | Method and device for detecting code review validity and electronic equipment | |
| CN113590484A (en) | Algorithm model service testing method, system, equipment and storage medium | |
| CN113051128B (en) | Power consumption detection method and device, electronic equipment and storage medium | |
| CN115080401A (en) | Automatic testing method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |