CN117539852A - Anti-cheating data reading method and device, electronic equipment and storage medium - Google Patents
Anti-cheating data reading method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117539852A CN117539852A CN202311498502.7A CN202311498502A CN117539852A CN 117539852 A CN117539852 A CN 117539852A CN 202311498502 A CN202311498502 A CN 202311498502A CN 117539852 A CN117539852 A CN 117539852A
- Authority
- CN
- China
- Prior art keywords
- cheating
- target
- table structure
- data
- difference
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000004140 cleaning Methods 0.000 claims abstract description 51
- 230000008569 process Effects 0.000 description 12
- 238000011144 upstream manufacturing Methods 0.000 description 12
- 230000008859 change Effects 0.000 description 10
- 238000012986 modification Methods 0.000 description 10
- 230000004048 modification Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 230000003247 decreasing effect Effects 0.000 description 3
- 238000012217 deletion Methods 0.000 description 3
- 230000037430 deletion Effects 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 238000007792 addition Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to a method, a device, electronic equipment and a storage medium for reading anti-cheating data, which are applied to a data anti-cheating system, wherein the data anti-cheating system comprises an anti-cheating server and a client, and the method comprises the following steps: determining a target data table to be read, and reading a target table structure of the target data table from a data warehouse; updating the anti-cheating table structure into a target anti-cheating table structure consistent with the target table structure under the condition that the cached anti-cheating table structure is inconsistent with the target table structure, wherein the anti-cheating table structure is the table structure of the target data table cached in the historical time period; performing anti-cheating cleaning on the target data table according to the target anti-cheating table structure to obtain a real target data table; and sending the real target data table to the client. Therefore, when anti-cheating cleaning is carried out on the data, the table structure can be updated in time, the efficiency and the accuracy of reading the anti-cheating data are improved, and the user experience is improved.
Description
Technical Field
The present disclosure relates to the field of data processing technologies, and in particular, to a method and apparatus for reading anti-cheating data, an electronic device, and a storage medium.
Background
In practical application, in the process of playing video by the video software, a database is used for storing data such as video playing amount and evaluation, and as the playing amount brings a certain benefit to a video uploading person, the possibility of cheating the data in the database may exist.
Based on this, in the prior art, there is often a anti-cheating system, which may include an anti-cheating module to anti-cheat process the data. Wherein the anti-cheating module is located downstream of the ETL (Extract transform load, data warehouse technology) and upstream of the business consumer (e.g., client). The anti-cheating module is used for carrying out anti-cheating processing on data in the data table, and when the table structure in the data warehouse is required to be modified, the modification is required manually, and the data updating flow is as follows: the feedback of the service using direction ETL needs to add a certain number of fields (such as fields a, b and c) in the data table, after the ETL adds the fields, the feedback anti-cheating module synchronously adds the fields a, b and c, then the table structure cached in the anti-cheating module is correspondingly modified manually by a person, task codes are manually modified to add the three fields a, b and c, and then the service using party can use the data table.
According to the method, when the table structure in the data warehouse of the anti-cheating system needs to be modified, the modification process needs a long time (at least more than one day), so that the problem that the data table structure is not updated timely exists, and the data read by the client cannot be updated synchronously with the ETL data in time, so that the user experience is affected.
Disclosure of Invention
The application provides a method, a device, electronic equipment and a storage medium for reading anti-cheating data, which are used for solving the technical problems that in the prior art, when a table structure in a data warehouse of an anti-cheating system needs to be modified, the modification process needs a long time (at least one day is needed), and the data table structure is not updated timely, so that data read by a client cannot be updated synchronously with ETL data in time, and user experience is affected.
In a first aspect, an embodiment of the present application provides a method for reading anti-cheating data, which is applied to a data anti-cheating system, where the data anti-cheating system includes an anti-cheating server and a client, and the method includes:
determining a target data table to be read, and reading a target table structure of the target data table from a data warehouse;
Updating the anti-cheating table structure into a target anti-cheating table structure consistent with the target table structure under the condition that the cached anti-cheating table structure is inconsistent with the target table structure, wherein the anti-cheating table structure is the table structure of the target data table cached in the historical time period;
performing anti-cheating cleaning on the target data table according to the target anti-cheating table structure to obtain a real target data table;
and sending the real target data table to the client.
As one possible implementation manner, the updating the anti-cheating table structure to a target anti-cheating table structure consistent with the target table structure includes:
determining a difference structure of the anti-cheating table structure and the target table structure;
determining an object code template corresponding to the difference structure from a plurality of code templates according to the difference structure;
determining an object code according to the difference structure and the object code template;
and executing the target code to merge the difference structure into the anti-cheating table structure to obtain the target anti-cheating table structure.
As one possible implementation manner, the determining the difference structure between the anti-cheating table structure and the target table structure includes:
Determining whether the number of fields of the anti-cheating table structure is consistent with the number of target fields of the target table structure;
and under the condition that the field quantity is inconsistent with the target field quantity, determining a difference field of the anti-cheating table structure and the target table structure, and determining the difference field as a difference structure of the anti-cheating table structure and the target table structure.
As one possible implementation manner, the determining the difference structure between the anti-cheating table structure and the target table structure includes:
determining whether the number of fields of the anti-cheating table structure is consistent with the number of target fields of the target table structure;
determining whether a difference field attribute inconsistent with the field attribute of the target table structure exists in the field attribute of the anti-cheating table structure under the condition that the field number is consistent with the target field number;
and determining the difference field attribute as a difference structure of the anti-cheating table structure and the target table structure in the case that the difference field attribute is determined to exist.
As one possible implementation manner, the determining the object code according to the difference structure and the object code template includes:
Determining field parameters corresponding to the difference structure;
and modifying the dynamic parameters in the target code template into the field parameters to obtain the target code.
As a possible implementation manner, the method further includes:
and under the condition that the cached anti-cheating table structure is consistent with the fields of the target table structure and the attribute of each field is consistent, performing anti-cheating cleaning on the target data table according to the anti-cheating table structure to obtain a real target data table.
As one possible implementation manner, before the determining the target data table to be read and reading the target table structure of the target data table from the data warehouse, the method further includes:
receiving a data table updating request sent by the client;
determining a first target data table to be updated, target operation and target field to be updated from the data table update request;
and updating the target field of the first target data table in the data warehouse according to the target operation.
In a second aspect, an embodiment of the present application provides an anti-cheating data reading device, which is applied to a data anti-cheating system, where the data anti-cheating system includes an anti-cheating server and a client, and the device includes:
The determining module is used for determining a target data table to be read and reading a target table structure of the target data table from a data warehouse;
the updating module is used for updating the anti-cheating table structure into a target anti-cheating table structure consistent with the target table structure under the condition that the cached anti-cheating table structure is inconsistent with the target table structure, wherein the anti-cheating table structure is the table structure of the target data table cached in the historical time period;
the cleaning module is used for performing anti-cheating cleaning on the target data table according to the target anti-cheating table structure to obtain a real target data table;
and the sending module is used for sending the real target data table to the client.
As a possible implementation manner, the updating module includes:
a first determining submodule, configured to determine a difference structure between the anti-cheating table structure and the target table structure;
the second determining submodule is used for determining an object code template corresponding to the difference structure from a plurality of code templates according to the difference structure;
a third determining submodule, configured to determine an object code according to the difference structure and the object code template;
And the execution sub-module is used for executing the target code to merge the difference structure into the anti-cheating table structure to obtain the target anti-cheating table structure.
As a possible implementation manner, the first determining submodule is specifically configured to:
determining whether the number of fields of the anti-cheating table structure is consistent with the number of target fields of the target table structure;
and under the condition that the field quantity is inconsistent with the target field quantity, determining a difference field of the anti-cheating table structure and the target table structure, and determining the difference field as a difference structure of the anti-cheating table structure and the target table structure.
As a possible implementation manner, the first determining submodule is specifically configured to:
determining whether the number of fields of the anti-cheating table structure is consistent with the number of target fields of the target table structure;
determining whether a difference field attribute inconsistent with the field attribute of the target table structure exists in the field attribute of the anti-cheating table structure under the condition that the field number is consistent with the target field number;
and determining the difference field attribute as a difference structure of the anti-cheating table structure and the target table structure in the case that the difference field attribute is determined to exist.
As a possible implementation manner, the third determining submodule is specifically configured to:
determining field parameters corresponding to the difference structure;
and modifying the dynamic parameters in the target code template into the field parameters to obtain the target code.
As a possible implementation manner, the apparatus further includes:
and the anti-cheating cleaning module is used for carrying out anti-cheating cleaning on the target data table according to the anti-cheating table structure under the condition that the cached anti-cheating table structure is consistent with the fields of the target table structure and the attribute of each field is consistent, so as to obtain a real target data table.
As a possible implementation manner, the apparatus further includes:
the receiving module is used for receiving a data table updating request sent by the client before the target data table to be read is determined and the target table structure of the target data table is read from a data warehouse;
the target determining module is used for determining a first target data table to be updated, target operation and target fields to be updated from the data table updating request;
and the field updating module is used for updating the target field of the first target data table in the data warehouse according to the target operation.
In a third aspect, an embodiment of the present application provides an electronic device, including: a processor and a memory, the processor being configured to execute a anti-cheating data reading program stored in the memory, to implement the anti-cheating data reading method of any one of the first aspects.
In a fourth aspect, an embodiment of the present application provides a storage medium, where one or more programs are stored, where the one or more programs are executable by one or more processors to implement the anti-cheating data reading method according to any one of the first aspects.
According to the technical scheme provided by the embodiment of the application, the target data table to be read is determined, the target table structure of the target data table is read from the data warehouse, the anti-cheating table structure is updated to be the target anti-cheating table structure consistent with the target table structure under the condition that the cached anti-cheating table structure is inconsistent with the target table structure, wherein the anti-cheating table structure is the table structure of the target data table cached in the historical time period, anti-cheating cleaning is carried out on the target data table according to the target anti-cheating table structure, so that a real target data table is obtained, and the real target data table is sent to the client. According to the technical scheme, when the anti-cheating cleaning operation is carried out on the data table, the cached anti-cheating table structure and the current table structure of the data table stored in the data warehouse are automatically updated synchronously, so that when the anti-cheating cleaning is carried out on the data, the table structure is updated in time, the efficiency and the accuracy of reading the anti-cheating data are improved, and the user experience is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, and it will be obvious to a person skilled in the art that other drawings can be obtained from these drawings without inventive effort.
One or more embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements, and in which the figures of the drawings are not to be taken in a limiting sense, unless otherwise indicated.
Fig. 1 is a schematic structural diagram of an anti-cheating system according to an embodiment of the present application;
FIG. 2 is a flowchart of an embodiment of a method for reading anti-cheating data according to an embodiment of the present application;
FIG. 3 is a flowchart of an embodiment of another anti-cheating data reading method according to an embodiment of the present application;
FIG. 4 is a flowchart of an embodiment of another anti-cheating data reading method according to an embodiment of the present application;
fig. 5 is a schematic flow chart of determining field changes according to an embodiment of the present application;
FIG. 6 is a block diagram of an embodiment of an anti-cheating data reading apparatus according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present application based on the embodiments herein.
The following disclosure provides many different embodiments, or examples, for implementing different structures of the invention. In order to simplify the present disclosure, components and arrangements of specific examples are described below. They are, of course, merely examples and are not intended to limit the invention. Furthermore, the present invention may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed.
In order to solve the technical problems that in the prior art, when a table structure in a data warehouse of an anti-cheating system needs to be modified, a modification process needs a long time (at least one day), and the data table structure is not updated timely, so that data read by a client cannot be updated synchronously with ETL data, and user experience is influenced.
In order to facilitate understanding of the anti-cheating data reading method provided by the application, an application scenario related to the application is described in the following.
Referring to fig. 1, a schematic structural diagram of an anti-cheating system according to an embodiment of the present application is provided. As shown in fig. 1, the anti-cheating system 10 may include: a anti-cheating server 11 and a client 12.
The anti-cheating server 11 may be one server or may be a cluster formed by a plurality of servers, which is not limited in the embodiment of the present application.
The client 12 may be a hardware device or software that supports a network connection to provide various network services. When the client 12 is hardware, it may be a variety of electronic devices supporting a display screen, including but not limited to smartphones, tablets, laptop computers, desktop computers, etc., only the client 12 is illustrated in fig. 1 as a desktop computer. When the client 12 is software, it may be installed in the electronic device enumerated above.
In the prior art, the anti-cheating server 11 may include a data warehouse and an anti-cheating module, where the data warehouse may be used to generate and store a corresponding data table for acquired data, and the data warehouse may be, for example, ETL (Extract, transform, load).
The anti-cheating module can be used for anti-cheating cleaning of data in a data table stored in a data warehouse, and false data in the data table, namely row data in the data table, can be cleaned through the existing anti-cheating method.
In practical application, when the client 12 reads the data in the data table stored in the data warehouse in the anti-cheating server 11, the anti-cheating server 11 may first perform anti-cheating cleaning on the data in the data table through the anti-cheating module, and then send the real data after the anti-cheating cleaning to the client 12, so as to ensure that the data read by the client 12 is the real data. The data in the data table stored in the data warehouse can be the data such as the playing amount, clicking rate, good evaluation and the like of any video in the video software, and also can be the data such as the reading amount of news in a website, and the embodiment of the application does not limit the data.
In addition, the client 12 may also change the structure of the data table stored in the data repository in the anti-cheating server 11 (i.e. modify the columns of the data table), for example, add or delete a certain field in the data table (e.g. video click rate), or change the attribute of a certain field in the data table, etc.
Based on this, when the anti-cheating server 11 receives a modification instruction of the data table in the data warehouse sent by the client 12, the structure of the data table can be directly modified.
However, when the anti-cheating module in the anti-cheating server 11 performs anti-cheating cleaning on the data table in the data warehouse, the anti-cheating cleaning is performed according to the table structure of the data table in the history period of the self-cache. In the prior art, after the table structure of the data table stored in the data warehouse is modified, the table structure cached by the anti-cheating module needs to be correspondingly modified manually, so that the anti-cheating module carries out anti-cheating cleaning according to the latest data table structure when carrying out anti-cheating cleaning on the data in the data table.
The method for manually modifying the table structure cached by the anti-cheating module is low in efficiency, slow in flow and easy to make mistakes, so that the data table structure in the anti-cheating module is not updated timely or makes mistakes easily, the anti-cheating module performs anti-cheating cleaning according to the original table structure or the wrong table structure, and therefore data read by a client cannot be updated timely in synchronization with data stored in a data warehouse or wrong data appear, and user experience is affected.
In this regard, the embodiment of the application provides a method for reading anti-cheating data, which can automatically update the table structure cached by the anti-cheating module and the table structure of the data warehouse synchronously, so that the table structure is updated in time when anti-cheating cleaning is performed on data, the efficiency and the accuracy of reading the anti-cheating data are improved, and the user experience is improved.
The anti-cheating data reading method provided in the present application is further explained with reference to the drawings in the following specific embodiments, which are not limiting of the embodiments of the present invention.
Referring to fig. 2, a flowchart of an embodiment of a method for reading anti-cheating data is provided in an embodiment of the present application. As an example, the flow shown in fig. 2 may be applied to a anti-cheating system, such as the anti-cheating system 10 shown in fig. 1, and further, it may be applied to the anti-cheating server 11 in the anti-cheating system 10. As shown in fig. 2, the process may include the steps of:
step 201, determining a target data table to be read, and reading a target table structure of the target data table from a data warehouse.
The target data table refers to a data table stored in a data warehouse to be read by the client.
The target table structure refers to a table structure corresponding to the target data table, and may include, but is not limited to: the fields included in the target data table, the attributes of each field, the order in which the fields included in the target data table correspond, and so on.
In an embodiment, the execution body of the embodiment of the present application may be an anti-cheating system, and further, the execution body of the embodiment of the present application may be an anti-cheating server in the anti-cheating system, which is not limited in this embodiment of the present application.
In an embodiment, the execution subject anti-cheating server of the embodiments of the present application may determine, when receiving a data read request sent by a client, a target data table to be read from the data read request.
In another embodiment, in order to prevent a problem of read data errors caused by inconsistent table structures of anti-cheating module caches in the anti-cheating servers and corresponding data tables in the data warehouse, the execution body of the embodiment of the present application may determine, as the target data table to be read, the data table whose table structure is changed when detecting that the table structure of any one of the data tables in the data warehouse is changed.
As an exemplary embodiment, when a modification request for a data table sent by a client is received and a corresponding data table is successfully modified according to the modification request, the modified data table is determined as a target data table to be read.
In addition, before determining the target data table to be read and reading the target table structure of the target data table from the data warehouse, the execution body of the embodiment of the present application may receive a data table update request sent by the client, and determine the target data table to be updated (for convenience of description, hereinafter referred to as a first target data table), the target operation (for example, operations of adding, deleting, changing, etc. the target data table) and the target field to be updated from the data table update request.
The target fields of the first target data table in the data warehouse may then be updated in accordance with the target operations described above.
In this embodiment of the present application, the data warehouse may be configured to store a data table and a table structure corresponding to the data table, and based on this, after determining a target data table to be read, the execution subject in this embodiment of the present application may read the table structure corresponding to the target data table (for convenience of description, hereinafter referred to as a target table structure) from the data warehouse.
Step 202, updating the anti-cheating table structure into a target anti-cheating table structure consistent with the target table structure under the condition that the cached anti-cheating table structure is inconsistent with the target table structure, wherein the anti-cheating table structure is a table structure of a target data table cached in a historical time period.
The anti-cheating table structure refers to a table structure stored by an anti-cheating module of the anti-cheating server, and can be a table structure stored by the anti-cheating module in a historical time period when the anti-cheating module processes a corresponding data table.
The anti-cheating module is provided with a cache area, a table structure of the data table is preset in the cache area, and based on the cache area, the anti-cheating module can conduct anti-cheating cleaning on the data table according to the cached table structure when conducting anti-cheating cleaning on the data table.
In this embodiment of the present application, in order to ensure that the table structure of the data table to be cleaned is synchronized with the table structure of the corresponding data table stored in the data warehouse, after determining the target table structure of the target data table, the execution body of the embodiment of the present application may first determine whether the cached anti-cheating table structure is consistent with the target table structure.
Optionally, under the condition that the cached anti-cheating table structure is inconsistent with the target table structure, the anti-cheating table structure can be updated to be the target anti-cheating table structure consistent with the target table structure, so that the anti-cheating module is ensured to conduct anti-cheating cleaning on the data table according to the table structure consistent with the table structure of the corresponding data table in the data warehouse.
As to how to update the anti-cheating table structure to a target anti-cheating table structure consistent with the target table structure, it is described below by the flow shown in fig. 3, which is not described in detail here.
And 203, performing anti-cheating cleaning on the target data table according to the target anti-cheating table structure to obtain a real target data table.
And 204, transmitting the real target data table to the client.
The following collectively describes steps 203 and 204:
the anti-cheating cleaning refers to anti-cheating cleaning of data in a data table by an anti-cheating module in an anti-cheating server according to an anti-cheating cleaning method in the prior art. The anti-cheating cleaning method herein is not described in detail in the embodiments of the present application.
The real target data table refers to a real data table after anti-cheating cleaning is performed on the target data table, and data stored in the real data table are all real.
In this embodiment of the present application, after updating the anti-cheating table structure cached in the anti-cheating module to a table structure consistent with the target table structure, the execution body of the embodiment of the present application may control the anti-cheating module to perform anti-cheating cleaning on the target data table according to the target anti-cheating table structure, so as to obtain a real target data table after anti-cheating cleaning.
And then, the real target data table can be sent to the client so that the client can acquire the real data of the target data table.
In addition, under the condition that the cached anti-cheating table structure is consistent with the target table structure, the target data table can be subjected to anti-cheating cleaning directly according to the anti-cheating table structure, so that a real target data table is obtained. The anti-cheating table structure and the target table structure are consistent, wherein the anti-cheating table structure and the target table structure can be consistent in terms of the fields included in the anti-cheating table structure and the target table structure, and the attribute of each field is consistent.
It should be noted that, when the above anti-cheating table structure is consistent with the fields included in the target table structure and the attribute of each field, and the sequence of the included fields is inconsistent, the anti-cheating table structure may not be updated because the contents included in the anti-cheating table structure and the target table structure are consistent.
According to the technical scheme provided by the embodiment of the application, the target data table to be read is determined, the target table structure of the target data table is read from the data warehouse, the anti-cheating table structure is updated to be the target anti-cheating table structure consistent with the target table structure under the condition that the cached anti-cheating table structure is inconsistent with the target table structure, wherein the anti-cheating table structure is the table structure of the target data table cached in the historical time period, anti-cheating cleaning is carried out on the target data table according to the target anti-cheating table structure, so that a real target data table is obtained, and the real target data table is sent to the client. According to the technical scheme, when the anti-cheating cleaning operation is carried out on the data table, the cached anti-cheating table structure and the current table structure of the data table stored in the data warehouse are automatically updated synchronously, so that when the anti-cheating cleaning is carried out on the data, the table structure is updated in time, the efficiency and the accuracy of reading the anti-cheating data are improved, and the user experience is improved.
Referring to fig. 3, a flowchart of an embodiment of another anti-cheating data reading method is provided in the embodiments of the present application. The flow shown in fig. 3 describes how the anti-cheating table structure is updated to a target anti-cheating table structure consistent with the target table structure, based on the flow shown in fig. 2. As shown in fig. 3, the process may include the steps of:
step 301, determining a difference structure between the anti-cheating table structure and the target table structure.
The above-mentioned difference structure refers to a portion where there is a difference between the structure of the anti-cheating table structure and the structure of the target table structure, which may be different fields or may be different attributes of any field, which is not limited in the embodiment of the present application.
In an embodiment, the execution body of the embodiment of the present application may determine whether the number of fields of the anti-cheating table structure and the number of target fields of the target table structure are consistent.
Alternatively, in the case where it is determined that the number of fields is inconsistent with the number of the target fields, since only one operation (for example, adding a field to the target data table or deleting a field, but not simultaneously deleting and adding a field) is performed when the data table in the data warehouse is updated, it may be explained that the target table structure at this time is updated and the operation of adding a field or deleting a field is performed.
Thus, a difference field of the anti-cheating table structure and the target table structure may be determined and determined as a difference structure of the anti-cheating table structure and the target table structure. It should be noted that, the difference field determined at this time is a difference field determined by taking the target table structure as a standard, for example, the anti-cheating table structure includes fields a, b, and c, the target table structure includes fields a, b, c, and d, and then the difference field is d. For another example, the anti-cheating table structure includes fields a, b, c, and d, and the target table structure includes fields a, b, and c, which indicate that the target table structure deletes field d, so-d may be determined as a difference field for distinction.
In contrast, in the case that it is determined that the number of fields of the anti-cheating table structure is identical to the number of fields of the target table structure, it is explained that the fields of the target table structure are not deleted or added at this time, but the attributes may be modified, so that it is possible to determine whether there is a difference field attribute inconsistent with the field attribute of the target table structure among the field attributes of the anti-cheating table structure, and in the case that it is determined that there is a difference field attribute, determine the difference field attribute as a difference structure of the anti-cheating table structure and the target table structure.
Step 302, determining an object code template corresponding to the difference structure from a plurality of code templates according to the difference structure.
Step 303, determining an object code according to the difference structure and the object code template.
Step 302 and step 303 are collectively described below:
the code template refers to a prestored SQL (Structured Query Language ) statement template, and parameters included in the code template can be dynamic parameters. The code templates may include statement templates for additions, deletions, modifications to the data table, and the like.
The above object code refers to code for merging the difference structure into the anti-cheating table structure.
In an embodiment, after the execution body in the embodiment of the present application determines the difference structure between the anti-cheating table structure and the target table structure, the target code template corresponding to the difference structure may be determined from the plurality of code templates according to the difference structure.
As an exemplary embodiment, in the case that the above-mentioned difference structure is a difference field, the corresponding target code template may be determined according to the identifier of the difference structure, for example, when the identifier is "-", it is explained that the number of fields of the anti-cheating table structure is greater than the number of fields of the target table structure at this time, and the difference field needs to be deleted, so the target code template is a deletion statement template (for example, "alter table antispam _ table replace columns $ { columns }). For another example, when the identifier is "+" or null, it is indicated that the number of fields of the anti-cheating table structure is smaller than the number of fields of the target table structure at this time, and a difference field needs to be added to the anti-cheating table structure, so that the target code template is an add statement template (e.g., "alter table antispam _ table add columns $ { columns }).
As another exemplary embodiment, in the case that the above-mentioned difference structure is a difference attribute, it is explained that the anti-cheating table structure and the target table structure have fields with inconsistent attributes, and thus, the determined target code template may be a modification statement template (e.g. "alter table antispam _table change $ { columns } string comment" i modified "").
Then, a corresponding object code may be determined based on the difference structure and the object code template.
As an exemplary embodiment, field parameters corresponding to the difference structure may be determined, and the dynamic parameters in the object code template may be modified to the field parameters of the difference structure, to obtain the object code.
For example, assuming that the difference structure is a difference field { (a, string), (b, string) }, and the object code template is an add statement template "alter table antispam _ table add columns $ { columns }, the dynamic parameters in the object code template are modified to the above-mentioned difference field parameters, and the object code" alter table antispam _ table add columns (astring, b string) can be obtained.
And 304, executing the target codes to merge the difference structure into the anti-cheating table structure to obtain the target anti-cheating table structure.
In the embodiment of the present application, since the above object code is used to merge the difference structure between the anti-cheating table structure and the object table structure into the anti-cheating table structure, the execution body of the embodiment of the present application may implement, by executing the above object code, to automatically merge the difference structure into the anti-cheating table structure, thereby implementing synchronous update of the anti-cheating table structure cached by the anti-cheating server and the object table structure stored in the data warehouse.
According to the technical scheme provided by the embodiment of the application, the difference structure of the anti-cheating table structure and the target table structure is determined, the target code template corresponding to the difference structure is determined from a plurality of code templates according to the difference structure, the target code is determined according to the difference structure and the target code template, and the target code is executed so as to merge the difference structure into the anti-cheating table structure, and the target anti-cheating table structure is obtained. According to the technical scheme, the target codes are determined according to the difference structures of the anti-cheating table structures and the target table structures and the preset code templates, and the difference structures are combined to the anti-cheating table structures by executing the target codes, so that the anti-cheating table structures cached by the anti-cheating server and the target table structures stored in the data warehouse are synchronously updated, the table structures are updated in time when the data are subjected to anti-cheating cleaning, the efficiency and the accuracy of reading the anti-cheating data are improved, and the user experience is improved.
Referring to fig. 4, a flowchart of an embodiment of another anti-cheating data reading method is provided in the embodiments of the present application. The flow shown in fig. 4 is illustrated by the anti-cheating module being located in the middle of the upstream ETL (data warehouse) and the downstream data consumer, the data flow being ETL- > anti-cheating- > service consumer. As shown in fig. 4, the flow may include the following:
firstly, when each anti-cheating task starts to run, firstly acquiring the structure of the upstream ETL Hive table data.
Second, compare the table structure of the redis cache to see what changes have occurred to which fields (field add, delete, attribute, or simply order).
And thirdly, processing the anti-cheating post-table structure according to the change of the upstream field. The upstream field increases: the anti-cheating table field is increased; upstream field deletion: deleting the anti-cheating table field; the upstream field changes the attribute: the anti-cheating list field changes the attribute; upstream adjustment field order or upstream no adjustment: the anti-cheating table field is not adjusted.
In an embodiment, whether the field is changed or not can be determined by the process shown in fig. 5 (only one of the increase and decrease is performed at a time, and not both are performed together), referring to fig. 5, a schematic process for determining the change of the field according to the embodiment of the present application is provided, as shown in fig. 5, where the process may include: firstly judging whether the total number of the content fields in the ETL and the cache is changed, if so, branching to the right, judging whether the content fields are increased or decreased, if so, comparing the difference, finding the increased field, adding the increased field into a post-cheating table (alter table antispam _ table add columns (a string, b string, c string)) to update the cache and then returning. If the field is reduced, the reduced field is found, the update cache is deleted (ALTER TABLE antispam _ table REPLACE COLUMNS (d STRING, e STRING)) in the post-cheating table and then return. If the field is not increased or decreased, whether the attributes of the field change is judged, if the attributes change, the cache is updated by changing the attributes in the anti-cheating table (alter table antispam _ table change a a string comment 'i modified'), and then return. Finally, if the field is not increased or decreased, the return is not modified directly.
And fourthly, processing dynamic parameters in the code according to the change of the upstream field. In the task of forming the table finally, there will be the operation of select x, y, z from etl_table, where x, y, z are the fields in the upstream etl table, if the fields in the etl table change, then this part of the anti-cheating task cannot be written to, and can only be transferred by means of dynamic parameters, for example: select $ { columns } from etl_table, thus ensuring that each update is timely, and that the field is not changed by re-wire tasks.
As for the specific parameter values of $ { columns }, the result of the decision according to fig. 5 is also obtained, for example, if three columns a, b, c are added, then $ { columns = "x, y, z, a, b, c". If one column x is reduced, $ { columns } = "y, z". If the table is unchanged or only changed in attribute or only the y, z order is exchanged, $ { columns } = "x, y, z".
And fifthly, a retry mechanism is added in the module, so that the situation that the anti-cheating post-table in other parallel tasks is changed, but the dynamic parameters in the cleaning task are not changed can be prevented.
According to the technical scheme provided by the embodiment of the application, in the cleaning step of the anti-cheating platform, because the anti-cheating cleaning is based on the ETL table, before the anti-cheating cleaning task is started, the service is started to see whether the upstream ETL table is updated, which fields (added or deleted or changed in attribute) are updated, and if not, the anti-cheating table and the task are not subjected to task operation. If so, automatically updating (adding or deleting or changing the attribute) the anti-cheating table, and carrying out parameter transmission on the anti-cheating cleaning task, the method of using parameter transmission also avoids updating the whole task on line each time of updating, realizes synchronous updating of the anti-cheating table structure cached by the anti-cheating server and the target table structure stored by the data warehouse, thereby realizing timely updating of the table structure when carrying out anti-cheating cleaning on the data, improving the efficiency and accuracy of reading the anti-cheating data and improving the user experience.
Referring to fig. 6, a block diagram of an embodiment of an anti-cheating data reading apparatus is provided in an embodiment of the present application. The apparatus shown in fig. 6 may be applied to the data anti-cheating system 10 shown in fig. 1, and the data anti-cheating system 10 may include an anti-cheating server, as well as a client. As shown in fig. 6, the apparatus may include:
a determining module 61, configured to determine a target data table to be read, and read a target table structure of the target data table from a data warehouse;
the updating module 62 is configured to update the anti-cheating table structure to a target anti-cheating table structure consistent with the target table structure when it is determined that the cached anti-cheating table structure is inconsistent with the target table structure, where the anti-cheating table structure is a table structure of the target data table cached in the historical time period;
the cleaning module 63 is configured to perform anti-cheating cleaning on the target data table according to the target anti-cheating table structure, so as to obtain a real target data table;
and a sending module 64, configured to send the real target data table to the client.
As one possible implementation, the updating module 62 includes:
a first determining submodule, configured to determine a difference structure between the anti-cheating table structure and the target table structure;
The second determining submodule is used for determining an object code template corresponding to the difference structure from a plurality of code templates according to the difference structure;
a third determining submodule, configured to determine an object code according to the difference structure and the object code template;
and the execution sub-module is used for executing the target code to merge the difference structure into the anti-cheating table structure to obtain the target anti-cheating table structure.
As a possible implementation manner, the first determining submodule is specifically configured to:
determining whether the number of fields of the anti-cheating table structure is consistent with the number of target fields of the target table structure;
and under the condition that the field quantity is inconsistent with the target field quantity, determining a difference field of the anti-cheating table structure and the target table structure, and determining the difference field as a difference structure of the anti-cheating table structure and the target table structure.
As a possible implementation manner, the first determining submodule is specifically configured to:
determining whether the number of fields of the anti-cheating table structure is consistent with the number of target fields of the target table structure;
Determining whether a difference field attribute inconsistent with the field attribute of the target table structure exists in the field attribute of the anti-cheating table structure under the condition that the field number is consistent with the target field number;
and determining the difference field attribute as a difference structure of the anti-cheating table structure and the target table structure in the case that the difference field attribute is determined to exist.
As a possible implementation manner, the third determining submodule is specifically configured to:
determining field parameters corresponding to the difference structure;
and modifying the dynamic parameters in the target code template into the field parameters to obtain the target code.
As a possible implementation, the apparatus further comprises (not shown in the figure):
and the anti-cheating cleaning module is used for carrying out anti-cheating cleaning on the target data table according to the anti-cheating table structure under the condition that the cached anti-cheating table structure is consistent with the fields of the target table structure and the attribute of each field is consistent, so as to obtain a real target data table.
As a possible implementation, the apparatus further comprises (not shown in the figure):
the receiving module is used for receiving a data table updating request sent by the client before the target data table to be read is determined and the target table structure of the target data table is read from a data warehouse;
The target determining module is used for determining a first target data table to be updated, target operation and target fields to be updated from the data table updating request;
and the field updating module is used for updating the target field of the first target data table in the data warehouse according to the target operation.
As shown in fig. 7, a schematic structural diagram of an electronic device according to an embodiment of the present application includes a processor 71, a communication interface 72, a memory 73 and a communication bus 74, where the processor 71, the communication interface 72, and the memory 73 complete communication with each other through the communication bus 74,
a memory 73 for storing a computer program;
in one embodiment of the present application, the processor 71 is configured to implement the anti-cheating data reading method provided in any one of the foregoing method embodiments when executing the program stored on the memory 73, where the method includes:
determining a target data table to be read, and reading a target table structure of the target data table from a data warehouse;
updating the anti-cheating table structure into a target anti-cheating table structure consistent with the target table structure under the condition that the cached anti-cheating table structure is inconsistent with the target table structure, wherein the anti-cheating table structure is the table structure of the target data table cached in the historical time period;
Performing anti-cheating cleaning on the target data table according to the target anti-cheating table structure to obtain a real target data table;
and sending the real target data table to the client.
The present application also provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the anti-cheating data reading method provided by any of the method embodiments described above.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
From the above description of embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus a general purpose hardware platform, or may be implemented by hardware. Based on such understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the related art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the method described in the respective embodiments or some parts of the embodiments.
It is to be understood that the terminology used herein is for the purpose of describing particular example embodiments only, and is not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "comprises," "comprising," "includes," "including," and "having" are inclusive and therefore specify the presence of stated features, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and/or groups thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring their performance in the particular order described or illustrated, unless an order of performance is explicitly stated. It should also be appreciated that additional or alternative steps may be used.
The foregoing is only a specific embodiment of the invention to enable those skilled in the art to understand or practice the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method for reading anti-cheating data, which is applied to a data anti-cheating system, wherein the data anti-cheating system comprises an anti-cheating server and a client, and the method comprises the following steps:
determining a target data table to be read, and reading a target table structure of the target data table from a data warehouse;
updating the anti-cheating table structure into a target anti-cheating table structure consistent with the target table structure under the condition that the cached anti-cheating table structure is inconsistent with the target table structure, wherein the anti-cheating table structure is the table structure of the target data table cached in the historical time period;
performing anti-cheating cleaning on the target data table according to the target anti-cheating table structure to obtain a real target data table;
and sending the real target data table to the client.
2. The method of claim 1, wherein the updating the anti-cheating table structure to a target anti-cheating table structure consistent with the target table structure comprises:
determining a difference structure of the anti-cheating table structure and the target table structure;
determining an object code template corresponding to the difference structure from a plurality of code templates according to the difference structure;
Determining an object code according to the difference structure and the object code template;
and executing the target code to merge the difference structure into the anti-cheating table structure to obtain the target anti-cheating table structure.
3. The method of claim 2, wherein the determining a difference structure of the anti-cheating table structure and the target table structure comprises:
determining whether the number of fields of the anti-cheating table structure is consistent with the number of target fields of the target table structure;
and under the condition that the field quantity is inconsistent with the target field quantity, determining a difference field of the anti-cheating table structure and the target table structure, and determining the difference field as a difference structure of the anti-cheating table structure and the target table structure.
4. The method of claim 2, wherein the determining a difference structure of the anti-cheating table structure and the target table structure comprises:
determining whether the number of fields of the anti-cheating table structure is consistent with the number of target fields of the target table structure;
determining whether a difference field attribute inconsistent with the field attribute of the target table structure exists in the field attribute of the anti-cheating table structure under the condition that the field number is consistent with the target field number;
And determining the difference field attribute as a difference structure of the anti-cheating table structure and the target table structure in the case that the difference field attribute is determined to exist.
5. The method of claim 2, wherein said determining an object code from said difference structure and said object code template comprises:
determining field parameters corresponding to the difference structure;
and modifying the dynamic parameters in the target code template into the field parameters to obtain the target code.
6. The method according to claim 1, wherein the method further comprises:
and under the condition that the cached anti-cheating table structure is consistent with the fields of the target table structure and the attribute of each field is consistent, performing anti-cheating cleaning on the target data table according to the anti-cheating table structure to obtain a real target data table.
7. The method of claim 1, wherein prior to said determining a target data table to be read and reading a target table structure of said target data table from a data warehouse, said method further comprises:
receiving a data table updating request sent by the client;
Determining a first target data table to be updated, target operation and target field to be updated from the data table update request;
and updating the target field of the first target data table in the data warehouse according to the target operation.
8. An anti-cheating data reading apparatus, applied to a data anti-cheating system, the data anti-cheating system comprising an anti-cheating server, and a client, the apparatus comprising:
the determining module is used for determining a target data table to be read and reading a target table structure of the target data table from a data warehouse;
the updating module is used for updating the anti-cheating table structure into a target anti-cheating table structure consistent with the target table structure under the condition that the cached anti-cheating table structure is inconsistent with the target table structure, wherein the anti-cheating table structure is the table structure of the target data table cached in the historical time period;
the cleaning module is used for performing anti-cheating cleaning on the target data table according to the target anti-cheating table structure to obtain a real target data table;
and the sending module is used for sending the real target data table to the client.
9. An electronic device, comprising: a processor and a memory, the processor being configured to execute a anti-cheating data reading program stored in the memory to implement the anti-cheating data reading method of any of claims 1-7.
10. A storage medium storing one or more programs executable by one or more processors to implement the anti-cheating data reading method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311498502.7A CN117539852A (en) | 2023-11-10 | 2023-11-10 | Anti-cheating data reading method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311498502.7A CN117539852A (en) | 2023-11-10 | 2023-11-10 | Anti-cheating data reading method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117539852A true CN117539852A (en) | 2024-02-09 |
Family
ID=89793204
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311498502.7A Pending CN117539852A (en) | 2023-11-10 | 2023-11-10 | Anti-cheating data reading method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117539852A (en) |
-
2023
- 2023-11-10 CN CN202311498502.7A patent/CN117539852A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11243981B2 (en) | Database replication based on data access scores | |
| US8868637B2 (en) | Page rendering for dynamic web pages | |
| US8554832B1 (en) | Server side user interface simulation | |
| US9383988B2 (en) | System and method for using directed acyclic graph (DAG) for application updates | |
| US10257110B2 (en) | Using a template to update a stack of resources | |
| US11755461B2 (en) | Asynchronous consumer-driven contract testing in micro service architecture | |
| US11397750B1 (en) | Automated conflict resolution and synchronization of objects | |
| CN112368682B (en) | Using caches for content verification and error remediation | |
| CN109614271B (en) | Method, device, equipment and storage medium for controlling consistency of multiple cluster data | |
| CN111190892A (en) | Method and device for processing abnormal data in data backfilling | |
| CN110795495A (en) | Data processing method and device, electronic equipment and computer readable medium | |
| CN110795494A (en) | Automatic testing method and device for synchronous and asynchronous cache data | |
| CN111338834A (en) | Data storage method and device | |
| US10067808B2 (en) | Nondeterministic operation execution environment utilizing resource registry | |
| CN117539852A (en) | Anti-cheating data reading method and device, electronic equipment and storage medium | |
| KR20180010442A (en) | User device for processing online authoring service and method for storing document thereof | |
| CN116739845A (en) | Contract generation method and device, electronic equipment and storage medium | |
| US12001458B2 (en) | Multi-cloud object store access | |
| CN113190236A (en) | HQL script checking method and device | |
| CN111596941B (en) | Application updating method, device, electronic equipment and medium | |
| CN118210595B (en) | Rule engine management system, method and related equipment based on edge cloud native | |
| US11582302B2 (en) | System and method for offline-first application development | |
| CN112632159B (en) | Database access control method and device, electronic equipment and storage medium | |
| US11948024B2 (en) | Automated dynamic payload testing of OData APIs | |
| CN112286970B (en) | Information processing method, system, computer device and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |