CN115858377A - Data testing system and method based on guest group management - Google Patents
Data testing system and method based on guest group management Download PDFInfo
- Publication number
- CN115858377A CN115858377A CN202211641734.9A CN202211641734A CN115858377A CN 115858377 A CN115858377 A CN 115858377A CN 202211641734 A CN202211641734 A CN 202211641734A CN 115858377 A CN115858377 A CN 115858377A
- Authority
- CN
- China
- Prior art keywords
- screening
- test
- metadata
- group
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention discloses a data testing system and method based on customer group management, which can perform secondary creation on a metadata resource library through a metadata management module to generate original metadata information with richer types and higher configurability degree, can construct dynamically configurable rule information through the metadata management module, avoid a fixed form which can be completed only by multiple development due to service change, support recursive logic operation or NAND, construct an operable execution engine to form complete operation expression capability, construct a customer group capable of being freely cut through a freely combinable relation component and a dynamically configurable rule component, realize the construction of the customer group under various different requirements, provide a testing method for customer group management, construct a quality guarantee system based on a target customer group, and ensure the transaction safety after testing.
Description
Technical Field
The invention belongs to the technical field of passenger group management and particularly relates to a data testing system and method based on passenger group management
Background
Customer group management, namely Customer group management, is an important ring in Customer relationship management (Customer relationship management), and aims to divide a Customer group by a portrait index according to different business scene requirements, so as to provide innovative personalized marketing and service technology for each Customer in the Customer group.
Most of the commercial customer group management systems use the application architecture of the microservice, and the customer group management adopts the configuration of the traditional configuration form, so the following problems exist in the existing customer group management technology:
1. the traditional guest group management uses the form to carry out configuration, the form configuration operation is inflexible, the complex logic relation (and or not) can not be configured, and the configuration display content of the form is not clear enough.
2. The method does not have the metadata management adaptation function, and cannot perform secondary creation on metadata resources, so that the original metadata information with richer types and higher configurable degree cannot be output.
3. Dynamically configurable rule information cannot be constructed, thereby complicating business changes.
4. The managed guest group data is not tested, and the quality of guest group management cannot be guaranteed.
Disclosure of Invention
In view of the above-mentioned deficiencies of the prior art, the present application provides a data testing system and method based on guest group management.
The application provides a data testing system based on guest group management, which comprises a metadata resource library, a metadata management module, a guest group management module and a target guest group data testing module;
the metadata resource library is used for storing predefined original metadata and metadata to be updated acquired from a target system;
the metadata management module is used for updating and managing the original metadata and the metadata to be updated;
the guest group management module is used for acquiring a metadata processing result from the metadata management module, performing screening assembly plug-in type configuration, guest group screening logic visual construction and recursive logic operation based on the metadata processing result, and performing screening logic construction and recursive processing through configuration conditions;
the target customer group data testing module is used for obtaining a target customer group through the customer group management module, performing plug-in type configuration of a testing component, visual building of testing logic and auditing and verifying of testing results on the target customer group, and completing testing/performing exception recording according to the testing results of target customer group data testing.
In some optional implementations of some embodiments, the metadata management module includes a data adaptation unit, a data type identification unit, a field management unit, a guest group information generation unit, and a filtering information generation unit;
the data adaptation unit is used for comparing the metadata to be updated with the original metadata and updating the original metadata according to a comparison result to obtain updated metadata;
the data type identification unit is used for classifying the updated metadata according to the data type to obtain a metadata classification result, and the metadata classification result comprises character metadata, date metadata and numerical metadata;
the field management unit is used for carrying out field extraction on the updated metadata to obtain a field extraction result, and the field extraction result comprises a field name, a field Chinese name and a field type;
the guest group information generating unit is used for performing combined analysis according to the metadata classification result and the metadata processing result of the field management unit, judging whether the corresponding metadata meets guest group conditions, and if so, generating corresponding guest group information;
and the screening information generating unit is used for performing combination analysis according to the metadata classification result and the metadata processing result of the field management unit, judging whether the corresponding metadata meets screening conditions, and if so, generating corresponding screening information.
In some optional implementation manners of some embodiments, the guest group management module includes a screening component plug-in configuration unit, a guest group screening logic visualization construction unit, and a recursive logic operation unit;
the screening component plug-in configuration unit is used for configuring a preset number of screening components according to the requirements on screening dimensions, and the screening components comprise a guest group rule information component, a preset guest group screening component and a relation component;
the guest group rule information component is used for selecting guest group information from the guest group information generating unit and selecting screening information from the screening information generating unit, and generating a guest group rule information table for screening guest groups after integrating selection results of the guest group information and the screening information;
the preset passenger group screening component is used for generating a preset passenger group screening information table for additionally screening the passenger groups according to the predefined passenger groups;
the relation component is used for configuring a relation operator for carrying out logical operation on the rule information table and the screening information table;
the customer group screening logic visualization construction unit is used for selecting corresponding customer group rule information from the customer group rule information table according to customer group screening requirements, selecting corresponding preset customer group screening information from the preset customer group screening information table, selecting corresponding relation operators from the relation components, and constructing a screening logic tree based on the rule information, the preset customer group screening information and the relation operators;
and the recursive logic operation unit is used for performing recursive operation on the screening logic tree built by the guest group screening logic visualization building unit to obtain a recursive result expression.
In some optional implementation manners of some embodiments, the target customer group data testing module includes a test component plug-in configuration unit, a test logic visualization construction unit, a test result auditing and verifying unit, and an updating configuration unit;
the test component plug-in type configuration unit is used for configuring a preset number of test components according to the requirement on test dimensionality, and the test components comprise a target guest group component, a test rule component and an event component;
the target customer group component is used for generating a target customer group information table according to the screened target customer group;
the test rule component is used for generating a test rule table according to a preset test screening rule;
the event component is used for generating a test event table according to a preset test screening event;
the test logic visual construction unit is used for selecting corresponding target customer group information from the customer group information table, selecting corresponding test rules from the test rule table, selecting corresponding test events from the test event table, and carrying out visual construction on the target customer group information, the test rules and the test events according to test requirements to obtain a target customer group data test flow;
the test result auditing and verifying unit is used for presetting an expected screening value, analyzing and comparing the test result of the target customer group data test flow according to the expected screening value, if the expected screening value is in accordance with the expected screening value, indicating that the test result is normal, and if the expected screening value is not in accordance with the expected screening value, indicating that the test result is abnormal, interrupting the test and recording to obtain a test result recording table;
and the updating configuration unit is used for configuring updating time and updating frequency for the target customer group data test process passing the verification and validation so as to perform closed-loop execution on the target customer group data test process.
In some optional implementations of some embodiments, the system further comprises a guest group management visualization interface and a target guest group data testing visualization interface;
the guest group management visual interface is used for loading the guest group management module and carrying out visual operation on a screening component plug-in configuration unit, a guest group screening logic visual construction unit and a recursive logic operation unit in the guest group management module;
the visual interface for target customer group data test is used for loading the target customer group data test module and visually operating a test assembly plug-in type configuration unit, a test logic visual building unit, a test result auditing and verifying unit and an updating and configuring unit in the target customer group data test module.
In a second aspect, the present application provides a data testing method based on guest group management, including the following steps:
acquiring predefined original metadata and collecting metadata to be updated from a target system;
managing and updating the original metadata and the metadata to be updated to obtain a metadata processing result;
constructing a screening logic tree according to the metadata processing result and a preset screening dimension, carrying out recursive operation on the screening logic tree in a recursive mode to obtain a recursive result expression, and obtaining a target guest group based on the recursive result expression;
and constructing a target guest group data test flow according to the target guest group and preset test conditions to obtain a test result, carrying out audit verification on the test result, and judging whether the test is passed or abnormal records are carried out through the audit verification.
In some optional implementations of some embodiments, the managing and updating the original metadata and the metadata to be updated, and the metadata processing result includes:
comparing the metadata to be updated with the original metadata, and updating the original metadata according to a comparison result to obtain updated metadata;
classifying the updated metadata according to the data type to obtain a metadata classification result, wherein the metadata classification result comprises character metadata, date metadata and numerical metadata;
performing field extraction on the updated metadata to obtain a field extraction result, wherein the field extraction result comprises a field name, a field Chinese name and a field type;
judging whether the corresponding metadata accords with the guest group condition or not according to the metadata classification result and the field extraction result, and if so, generating corresponding guest group information;
judging whether the corresponding metadata accords with screening conditions or not according to the metadata classification result and the field extraction result, and if so, generating corresponding screening information;
and integrating the guest group information and the screening information to obtain a metadata processing result.
In some optional implementation manners of some embodiments, the constructing a filtering logic tree according to the metadata processing result and a preset filtering dimension, performing recursive operation on the filtering logic tree in a recursive manner to obtain a recursive result expression, and obtaining a target guest group based on the recursive result expression includes:
generating a guest group rule information table for screening guest groups according to the selection results of the guest group information and the screening information;
generating a preset passenger group screening information table for additionally screening the passenger groups through the predefined passenger groups;
selecting a relation operator for carrying out relation logic operation;
selecting corresponding guest group rule information from the guest group rule information table according to guest group screening requirements, selecting corresponding preset guest group screening information from the preset guest group screening information table, selecting corresponding relation operators from the relation components, and building a screening logic tree based on the rule information, the preset guest group screening information and the relation operators;
and carrying out recursion operation on the screening logic tree to obtain a recursion result expression, and obtaining a target guest group based on the recursion result expression.
In some optional implementations of some embodiments, the recursive operation is performed with the bottom layer up, and the layer-by-layer relation operation is performed.
In some optional implementation manners of some embodiments, the target guest group data test process is constructed according to the target guest group and preset test conditions to obtain a test result, the test result is subjected to audit verification, and the test is judged to pass/abnormal records are performed through the audit verification.
Generating a target customer group information table according to the screened target customer group;
generating a test rule table according to a preset test screening rule;
generating a test event table according to a preset test screening event;
selecting corresponding target guest group information from the guest group information table, selecting corresponding test rules from the test rule table, selecting corresponding test events from the test event table, and building a target guest group data test flow for the target guest group information, the test rules and the test events according to test requirements;
and presetting an expected screening value, analyzing and comparing the test result of the target customer group data test flow according to the expected screening value, if the expected screening value is in line with the expected screening value, indicating that the test result is normal, passing the audit verification, and if the expected screening value is not in line with the expected screening value, indicating that the test result is abnormal, interrupting the test and recording to obtain a test result recording table.
The invention has the beneficial effects that:
1. the metadata management module can be used for performing secondary creation on the metadata resource library to generate original metadata information with richer types and higher configurable degree.
2. The dynamic configurable rule information can be constructed through the metadata management module, and the fixed form which can be completed only by being developed for multiple times due to service change is avoided.
3. The system supports recursive logic operation or NAND, an operable execution engine is constructed, the automatic engine processing is carried out from the front end to the back end, drag and drag operations are carried out by combining a guest group management visual interface through the configuration of 'AND', 'OR' or 'not' (the 'AND' is the relation component 'and meets the requirement', 'OR' is the relation component 'or the component', and 'not' is the relation component 'rejects') relation components, the three relation components are freely combined to form complete operation expression capability, a guest group capable of being freely cut can be constructed through the freely-combined relation component and the dynamically-configured rule component, and the construction of the guest group under various different requirements can be realized.
4. The method for testing the passenger group management is provided, and a quality assurance system based on the target passenger group is established.
Drawings
FIG. 1 is a system block diagram of the present invention.
FIG. 2 is a diagram of backend data processing to build the guest population screening logic.
Fig. 3 is a schematic diagram of guest group management performed on a guest group management visualization interface.
Fig. 4 is a schematic diagram of a target customer group test performed on a target customer group data test visualization interface.
FIG. 5 is a flow chart of the method of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The first aspect of the application provides a data testing system based on guest group management, which comprises a metadata resource library, a metadata management module, a guest group management module and a target guest group data testing module;
the metadata resource library is used for storing predefined original metadata and metadata to be updated acquired from a target system;
the metadata management module is used for updating and managing the original metadata and the metadata to be updated;
in some optional implementations of some embodiments, the metadata management module includes a data adaptation unit, a data type identification unit, a field management unit, a guest group information generation unit, and a filtering information generation unit;
the data adaptation unit is used for comparing the metadata to be updated with the original metadata and updating the original metadata according to a comparison result to obtain updated metadata;
the method comprises the steps of comparing a theme table name of original metadata with a table name of metadata to be updated, if the names are not consistent, indicating that the metadata to be updated is unreported metadata, loading table data content of the metadata to be updated and then merging the table data content of the original metadata to obtain updated metadata, managing a theme table, wherein the management of the theme table comprises defining the table name, an online library, data loading frequency, validity and table description, and the theme table is derived from a basic table of a hive data warehouse or other data platforms.
The data type identification unit is used for classifying the updated metadata according to the data type to obtain a metadata classification result, and the metadata classification result comprises character metadata, date metadata and numerical metadata;
the field management unit is used for carrying out field extraction on the updated metadata to obtain a field extraction result, and the field extraction result comprises a field name, a field Chinese name and a field type;
the data type identification unit and the field management unit are integrated to manage the table fields, and the table field management is used for defining field basic information, field operational characters, field component types, dynamic verification rules and verification mismatching prompts. The field basic information supports two broad categories: one type is a base field type and one type is a derived field type.
The field basic information contained when the field type is the basic field type includes: basic fields, english names, chinese names, data types, effective or not and field description, wherein the basic fields are from table fields of a subject table, and each basic information is displayed in a reverse mode after a certain table field is selected through a drop-down box
The field basic information contained when the field type is a derived field type includes: the method comprises the following steps of field English name, field Chinese name, data type, validity or not, derived field calculation rule, reference field and field description, wherein the derived field supports the following data types: numerical type, character type, date type, type of type; the derived field calculation rule is a segment of operable SQL expression, and a new derived field is obtained by using the field in the reference field for operation, such as calculating the constellation through the birth date; the field operators are: equal to, match, contain, reject, range, greater than or equal to, less than or equal to, not match.
The field component types are: a numerical value input box, a numerical value range box, a drop-down radio box, a drop-down multi-selection box, a date control, a date range control, a character input box, a horizontal tiling radio box, a horizontal tiling multi-selection box, a popup box and a data period selection box; different field component types may be configured with different validation rules and component association codes (setting different code types).
Code type: code type data is also derived from metadata, such as certain enumerated type values.
The guest group information generating unit is used for performing combined analysis according to the metadata classification result and the metadata processing result of the field management unit, judging whether the corresponding metadata meets guest group conditions, and if so, generating corresponding guest group information;
and the screening information generating unit is used for performing combination analysis according to the metadata classification result and the metadata processing result of the field management unit, judging whether the corresponding metadata meets screening conditions, and if so, generating corresponding screening information.
For example: the table field contents of the updated metadata are as follows:
the client information which accords with the client condition and the screening condition simultaneously can be used as client information and also can be used as screening information, and the generated client information comprises: gender, card logout code, card activation days, card checking time, card activation date, card issuing (changing) reason code, mobile phone number, loyalty customers and card validity, etc., and the generated screening information comprises: gender, card logout code, number of card activation days, and date of card activation, etc.
The guest group management module is used for acquiring a metadata processing result from the metadata management module, performing screening component plug-in configuration, guest group screening logic visual construction and recursive logic operation based on the metadata processing result, and performing screening logic construction and recursive processing through configuration conditions;
further, the guest group management also comprises guest group definition and configuration, guest group base table parameterization, guest group processing cache, external list management and guest group insights;
the guest group definition and configuration comprises three modes of defining guest group management basic field information, operators such as logic operators and relational operators and guest group management;
the processing modes of the guest group management technology comprise three modes, including a wide table mode, a narrow table mode and a combined mode;
the wide table mode is used for screening condition splicing treatment (where condition);
the narrow table mode is used for splicing processing (sub-query) in a label-like mode;
the combined mode is used for processing association, intersection, union and difference among the passenger groups;
the guest group bottom table parameterization comprises bottom layer wide table parameterization, bottom layer narrow table parameterization, and non-standardized processing between guest groups. And the bottom layer wide table parameterization is realized, and the third-party user can operate the guest group management function by only replacing the corresponding bottom layer table name without code adjustment.
The bottom layer narrow table parameterization exists, the narrow table definition has personalized processing, data personnel are provided for narrow table processing, narrow table information is provided for configuration, and development processing of an interface part is reduced.
And the inter-guest group and non-standardized processing, and the guest group table definition is defined by guest group management, so that the inter-guest group processing is specified by adopting a standard format.
The guest group processing cache comprises guest group rule information JSON processing, guest group rule information JSON to SQL cache processing, and changes processing after creation exists among guest groups, so that the adjustment of the SQL cache when the guest groups change is clear.
And the external list management is used for providing a complete flow of template downloading, analysis, warehousing and the like.
And the customer group insights are used for providing a customer group insights entrance and performing API processing on the insights real-time query interface.
In some optional implementation manners of some embodiments, the guest group management module includes a screening component plug-in configuration unit, a guest group screening logic visualization construction unit, and a recursive logic operation unit;
the screening component plug-in configuration unit is used for configuring screening components with preset quantity according to the requirement on screening dimensionality, and the screening components comprise a customer group rule information component, a preset customer group screening component and a relation component, and further comprise a white list, a black list, a loss of credit list and a post-strategy standard list;
wherein, screening subassembly plug-in configuration unit can then select to increase/reduce the screening subassembly according to the demand of screening dimension, and the plug-in configuration of screening subassembly explains: the relation components comprise ' and meet ' or meet ' and ' reject ' and the like, rule information contained in the customer base rule information component comprises marital conditions, whether APP is bound with cards, the number of days of validity of certificates, card products, whether star customers or not and the like, all the components can be dragged and dropped into a customer base management visual interface in a mode of pulling and dragging pointers of a user, all the components are associated through association points on the pointers of the user, and finally the rule information is associated through the relation components to form an integral operator to calculate the customer base.
The guest group rule information component is used for selecting guest group information from the guest group information generating unit and selecting screening information from the screening information generating unit, and generating a guest group rule information table for screening guest groups after integrating selection results of the guest group information and the screening information;
the content of the guest group rule information table generated based on the guest group information and the screening information includes: gender, card logout codes, card activation days, card checking time, card activation date, card issuing (exchanging) reason codes, mobile phone numbers, loyalty customers, validity of cards and the like, and can be selected from the codes according to the requirement of dimension screening, so that the content of the corresponding customer group rule information table is customized.
The preset passenger group screening component is used for generating a preset passenger group screening information table for additionally screening the passenger groups according to the predefined passenger groups;
the preset guest group screening information table is composed of existing guest groups constructed based on predefined original metadata before accessing a third-party system.
The relation component is used for configuring a relation operator for carrying out logical operation on the rule information table and the screening information table;
the relationship component comprises three major operational relationship components: and satisfies a relation component, or eliminates a relation component, wherein the relation component corresponds to a unit in the sql operation, or satisfies an inner join in the sql operation, and the elimination relation component corresponds to a not exists or a not in the sql operation.
The customer group screening logic visualization construction unit is used for selecting corresponding customer group rule information from the customer group rule information table according to customer group screening requirements, selecting corresponding preset customer group screening information from the preset customer group screening information table, selecting corresponding relation operators from the relation components, and constructing a screening logic tree based on the rule information, the preset customer group screening information and the relation operators;
the building of the guest group screening logic can be built through a relationship component and rule information on the basis of a preset guest group screening information table, or can be built through the relationship component and the rule information directly, as shown in fig. 2, the specific process is that a Json format expression is generated during storage through the relationship component and the rule information on a guest group screening logic visual building unit, the Json format expression is converted into related sql by a background and pushed to a big data system, the big data system runs the sql to obtain actual guest group data, and finally the actual guest group data is pushed to an application system for storage;
and the recursion logic operation unit is used for carrying out recursion operation on the screening logic tree built by the guest group screening logic visual building unit to obtain a recursion result expression.
Wherein the recursive logical operation comprises an operation node and an operation node depth;
and (3) operation nodes: one operation node comprises a relation component and a plurality of rule information
Calculating the node depth: how many relationship components are traversed from the beginning to the current node is the depth of the node (containing the current node number).
The operation rule of the recursive logic operation is as follows: the method comprises the steps that operation is always started from an operation node with the largest depth, if a plurality of operation nodes with the same depth exist, operation is sequentially carried out according to the sequence, when the node is operated, a single rule information component is operated, each rule information component generates a temporary process table, all rule information under the node is operated according to a corresponding relation operation rule of the node to obtain a result of the operation node, after the operation of the node is finished, corresponding relation operation is carried out on the node and other nodes with the same level depth, after the operation of the node with the same level depth is finished, the current depth is reduced by 1, after the operation is finished, the operation is carried out on the node with the depth reduced by 2, the operation is carried out until the depth is 0 (the node is reached), the whole operation process is finished, and therefore a target customer group is obtained.
A specific calculation process is illustrated with reference to fig. 3, for the guest group configuration of fig. 3, during the calculation, the rule information component [ whether APP binds to card or not ] is calculated first, the component calculates and stores the data [ APP binds to card condition is yes ] in the first temporary process table (assumed as T1), then calculates the rule information component [ certificate type is equal to 0102], the component calculates and stores the data [ certificate type is equal to 0102] in the second temporary process table (assumed as T2), then performs or operation on T1 and T2, that is, union operation, obtains the third temporary process table (assumed as T3), at this time, deletes the first and second temporary process tables T1 and T2, then calculates "remove" [ star-level client is equal to no ] and the rule information component, a component that discards (not in) data of no star customers into a fourth temporary process table (assumed to be T4), a component that calculates and stores data of marriage status condition of marriage as marriage into a fifth temporary process table (assumed to be T5), a component that calculates and stores rule information of card product of jingdong LUS affiliation card into a sixth temporary process table (assumed to be T6), a component that calculates and stores data of card product of jingdong LUS affiliation card into a sixth temporary process table (assumed to be T6), and a component that deletes the third-sixth temporary process tables T3, T4, T5, T6 and calculates (inner join) into a final group result, and deletes the third-sixth temporary process tables T3, T4, T5, T6 after obtaining the final group result, the final guest group result is the target guest group we use for subsequent testing.
The target customer group data testing module is used for obtaining a target customer group through the customer group management module, performing plug-in type configuration of a testing component, visual building of testing logic and auditing and verifying of testing results on the target customer group, and completing testing/performing exception recording according to the testing results of target customer group data testing.
In some optional implementation manners of some embodiments, the target customer group data testing module includes a test component plug-in configuration unit, a test logic visualization construction unit, a test result auditing and verifying unit, and an updating configuration unit;
the test component plug-in configuration unit is used for configuring a preset number of test components according to the requirement on test dimensionality, and the test components comprise a target guest group component, a test rule component and an event component;
the configuration rules of the testing component plug-in configuration unit and the screening component plug-in configuration unit are similar, the testing component plug-in configuration unit can increase/decrease testing components according to the requirement of testing dimensionality, if the requirement on the precision of the testing dimensionality is higher, more testing components can be configured, and if the requirement on the precision of the testing dimensionality is lower, the operation efficiency needs to be ensured, the configuration of the testing components can be reduced;
the target customer group component is used for generating a target customer group information table according to the screened target customer group;
the target guest group information table is a final guest group result (namely a target guest group) calculated by the guest group management module through recursive logical operation, the guest group management module can obtain a plurality of target guest groups through different guest group configurations, the target guest group component records the target guest groups calculated in the guest group management module to form the target guest group information table, and the target guest group data test module selects and tests the target guest groups in the target guest group information table according to test requirements.
The test rule component is used for generating a test rule table according to a preset test screening rule;
as shown in fig. 4, the test screening rule includes condition splitting, basic event, combinatorial screening, and AB test, and a corresponding test rule component is selected according to a requirement set up by the test logic to perform screening test on the target customer group, for example, the condition splitting may be set as male and female splitting, and users in the target customer group are subjected to screening test according to gender conditions.
The event component is used for generating a test event table according to a preset test screening event;
as shown in fig. 4, the test screening event may be a customized test event, taking a target guest group after diversion of males and females as an example, the test event may be that a single consumption on the day is 1000 yuan/5 consumptions completed on the day, the customized test event is named as 9-2 test, and the test screening is performed on the male target guest group and the female target guest group after diversion of males and females respectively.
The test logic visual construction unit is used for selecting corresponding target customer group information from the customer group information table, selecting corresponding test rules from the test rule table, selecting corresponding test events from the test event table, and carrying out visual construction on the target customer group information, the test rules and the test events according to test requirements to obtain a target customer group data test flow;
the testing logic is built by a target customer group component, a testing rule component and an event component, each component is dragged and dropped into a data testing visual interface of the target customer group based on a user's' pulling 'and' dragging 'pointer mode, all components are associated through association points on the user's 'pulling' and 'dragging' components, and finally a target customer group testing process is formed.
The test result auditing and verifying unit is used for presetting an expected screening value, analyzing and comparing the test result of the target customer group data test flow according to the expected screening value, if the expected screening value is in accordance with the expected screening value, indicating that the test result is normal, and if the expected screening value is not in accordance with the expected screening value, indicating that the test result is abnormal, interrupting the test and recording to obtain a test result recording table;
after the target customer group test flow is constructed and tested, the test result auditing and verifying unit is arranged before the test result is obtained, the test result of the target customer group test flow is audited, an expected screening value is preset, the test result of the target customer group data test flow is analyzed and compared according to the expected screening value, if the expected screening value is met, the test result is normal, the current test result of the target customer group test flow can be stored and used for a specific marketing strategy through auditing and verifying, if the expected screening value is not met, the test result is abnormal, the test is interrupted and recorded, a test result record table is obtained, abnormal nodes or abnormal data sources are recorded in the test result record table, the abnormal nodes or the abnormal data sources are traced, and the transaction safety of the subsequent marketing strategy is ensured.
And the updating configuration unit is used for configuring updating time and updating frequency for the target customer group data test process passing the verification and validation so as to perform closed-loop execution on the target customer group data test process.
In some optional implementations of some embodiments, the system further comprises a guest group management visualization interface and a target guest group data testing visualization interface;
the guest group management visual interface is used for loading the guest group management module and carrying out visual operation on a screening component plug-in configuration unit, a guest group screening logic visual construction unit and a recursive logic operation unit in the guest group management module;
as shown in fig. 3, the screening component plug-in configuration unit, the guest group screening logic visualization construction unit, and the recursive logic operation unit may be visually operated.
The target customer group data test visual interface is used for loading the target customer group data test module and carrying out visual operation on a test assembly plug-in type configuration unit, a test logic visual building unit, a test result auditing and verifying unit and an updating and configuring unit in the target customer group data test module.
As shown in fig. 4, a plug-in configuration unit of a test component, a visual building unit of test logic, a verification unit of verification of test results, and an update configuration unit in the target customer group data test module may be visually operated.
In a second aspect, the present application provides a data testing method based on guest group management, including the following steps:
s100: acquiring predefined original metadata and collecting metadata to be updated from a target system;
s200: managing and updating the original metadata and the metadata to be updated to obtain a metadata processing result;
in some optional implementations of some embodiments, the managing and updating the original metadata and the metadata to be updated, and the metadata processing result includes:
comparing the metadata to be updated with the original metadata, and updating the original metadata according to a comparison result to obtain updated metadata;
classifying the updated metadata according to the data type to obtain a metadata classification result, wherein the metadata classification result comprises character metadata, date metadata and numerical metadata;
performing field extraction on the updated metadata to obtain a field extraction result, wherein the field extraction result comprises a field name, a field Chinese name and a field type;
judging whether the corresponding metadata accords with the guest group condition or not according to the metadata classification result and the field extraction result, and if so, generating corresponding guest group information;
judging whether the corresponding metadata accords with screening conditions or not according to the metadata classification result and the field extraction result, and if so, generating corresponding screening information;
and integrating the guest group information and the screening information to obtain a metadata processing result.
S300: constructing a screening logic tree according to the metadata processing result and a preset screening dimension, carrying out recursive operation on the screening logic tree in a recursive mode to obtain a recursive result expression, and obtaining a target guest group based on the recursive result expression;
in some optional implementation manners of some embodiments, the constructing a filtering logic tree according to the metadata processing result and a preset filtering dimension, performing recursive operation on the filtering logic tree in a recursive manner to obtain a recursive result expression, and obtaining a target guest group based on the recursive result expression includes:
generating a guest group rule information table for screening guest groups according to the selection result of the guest group information and the screening information;
generating a preset passenger group screening information table for additionally screening the passenger group through a predefined passenger group;
selecting a relation operator for carrying out relation logic operation;
selecting corresponding guest group rule information from the guest group rule information table according to guest group screening requirements, selecting corresponding preset guest group screening information from the preset guest group screening information table, selecting corresponding relation operators from the relation components, and building a screening logic tree based on the rule information, the preset guest group screening information and the relation operators;
and carrying out recursion operation on the screening logic tree to obtain a recursion result expression, and obtaining a target guest group based on the recursion result expression.
In some optional implementations of some embodiments, the recursive operation is performed with the bottom layer up, and the layer-by-layer relation operation is performed.
S400: and constructing a target guest group data test flow according to the target guest group and preset test conditions to obtain a test result, carrying out audit verification on the test result, and judging whether the test is passed or abnormal records are carried out through the audit verification.
In some optional implementation manners of some embodiments, the target guest group data test process is constructed according to the target guest group and preset test conditions to obtain a test result, the test result is subjected to audit verification, and the test is judged to pass or abnormal records are judged through the audit verification.
Generating a target passenger group information table according to the screened target passenger groups;
generating a test rule table according to a preset test screening rule;
generating a test event table according to a preset test screening event;
selecting corresponding target guest group information from the guest group information table, selecting corresponding test rules from the test rule table, selecting corresponding test events from the test event table, and building a target guest group data test flow for the target guest group information, the test rules and the test events according to test requirements;
and presetting an expected screening value, analyzing and comparing the test result of the target customer group data test flow according to the expected screening value, if the expected screening value is in line with the expected screening value, indicating that the test result is normal, passing the verification, and if the expected screening value is not in line with the expected screening value, indicating that the test result is abnormal, interrupting the test and recording to obtain a test result recording table.
The above is only a preferred embodiment of the present invention, and it should be noted that several modifications and improvements made by those skilled in the art without departing from the technical solution should also be considered as falling within the scope of the claims.
Claims (10)
1. Data test system based on visitor's crowd management, its characterized in that: the system comprises a metadata resource library, a metadata management module, a guest group management module and a target guest group data test module;
the metadata resource library is used for storing predefined original metadata and metadata to be updated acquired from a target system;
the metadata management module is used for updating and managing the original metadata and the metadata to be updated;
the guest group management module is used for acquiring a metadata processing result from the metadata management module, performing screening assembly plug-in type configuration, guest group screening logic visual construction and recursive logic operation based on the metadata processing result, and performing screening logic construction and recursive processing through configuration conditions;
the target customer group data testing module is used for acquiring a target customer group through the customer group management module, performing plug-in type configuration of a testing assembly, visual construction of testing logic and verification of testing results on the target customer group, and completing testing/performing exception recording according to the testing results of the target customer group data testing.
2. The system of claim 1, wherein: the metadata management module comprises a data adaptation unit, a data type identification unit, a field management unit, a guest group information generation unit and a screening information generation unit;
the data adaptation unit is used for comparing the metadata to be updated with the original metadata and updating the original metadata according to a comparison result to obtain updated metadata;
the data type identification unit is used for classifying the updated metadata according to the data type to obtain a metadata classification result, and the metadata classification result comprises character metadata, date metadata and numerical metadata;
the field management unit is used for carrying out field extraction on the updated metadata to obtain a field extraction result, and the field extraction result comprises a field name, a field Chinese name and a field type;
the guest group information generating unit is used for performing combined analysis according to the metadata classification result and the metadata processing result of the field management unit, judging whether the corresponding metadata meets guest group conditions, and if so, generating corresponding guest group information;
and the screening information generating unit is used for performing combination analysis according to the metadata classification result and the metadata processing result of the field management unit, judging whether the corresponding metadata meets screening conditions, and if so, generating corresponding screening information.
3. The system of claim 2, wherein: the guest group management module comprises a screening component plug-in configuration unit, a guest group screening logic visualization building unit and a recursive logic operation unit;
the screening component plug-in configuration unit is used for configuring a preset number of screening components according to the requirements on screening dimensions, and the screening components comprise a guest group rule information component, a preset guest group screening component and a relation component;
the guest group rule information component is used for selecting guest group information from the guest group information generating unit and selecting screening information from the screening information generating unit, and generating a guest group rule information table for screening guest groups after integrating selection results of the guest group information and the screening information;
the preset passenger group screening component is used for generating a preset passenger group screening information table for additionally screening the passenger groups according to the predefined passenger groups;
the relation component is used for configuring a relation operator for carrying out logical operation on the rule information table and the screening information table;
the customer group screening logic visualization construction unit is used for selecting corresponding customer group rule information from the customer group rule information table according to customer group screening requirements, selecting corresponding preset customer group screening information from the preset customer group screening information table, selecting corresponding relation operators from the relation components, and constructing a screening logic tree based on the rule information, the preset customer group screening information and the relation operators;
and the recursion logic operation unit is used for carrying out recursion operation on the screening logic tree built by the guest group screening logic visual building unit to obtain a recursion result expression.
4. The system of claim 3, wherein: the target customer group data testing module comprises a testing component plug-in type configuration unit, a testing logic visual building unit, a testing result auditing and verifying unit and an updating and configuring unit;
the test component plug-in configuration unit is used for configuring a preset number of test components according to the requirement on test dimensionality, and the test components comprise a target guest group component, a test rule component and an event component;
the target guest group component is used for generating a target guest group information table according to the screened target guest groups;
the test rule component is used for generating a test rule table according to a preset test screening rule;
the event component is used for generating a test event table according to a preset test screening event;
the test logic visualization building unit is used for selecting corresponding target customer group information from the customer group information table, selecting corresponding test rules from the test rule table, selecting corresponding test events from the test event table, and performing visualization building on the target customer group information, the test rules and the test events according to test requirements to obtain a target customer group data test flow;
the test result auditing and verifying unit is used for presetting an expected screening value, analyzing and comparing the test result of the target customer group data test flow according to the expected screening value, if the expected screening value is in accordance with the expected screening value, indicating that the test result is normal, and if the expected screening value is not in accordance with the expected screening value, indicating that the test result is abnormal, interrupting the test and recording to obtain a test result recording table;
and the updating configuration unit is used for configuring updating time and updating frequency for the target customer group data test process passing the verification and validation so as to perform closed-loop execution on the target customer group data test process.
5. The system according to any one of claims 1-4, wherein: the system also comprises a guest group management visual interface and a target guest group data test visual interface;
the guest group management visual interface is used for loading the guest group management module and carrying out visual operation on a screening component plug-in configuration unit, a guest group screening logic visual building unit and a recursive logic operation unit in the guest group management module;
the target customer group data test visual interface is used for loading the target customer group data test module and carrying out visual operation on a test assembly plug-in type configuration unit, a test logic visual building unit, a test result auditing and verifying unit and an updating and configuring unit in the target customer group data test module.
6. The data testing method based on the guest group management is characterized in that: the method comprises the following steps:
acquiring predefined original metadata and collecting metadata to be updated from a target system;
managing and updating the original metadata and the metadata to be updated to obtain a metadata processing result;
constructing a screening logic tree according to the metadata processing result and a preset screening dimension, carrying out recursive operation on the screening logic tree in a recursive mode to obtain a recursive result expression, and obtaining a target guest group based on the recursive result expression;
and constructing a target guest group data test flow according to the target guest group and preset test conditions to obtain a test result, carrying out audit verification on the test result, and judging whether the test is passed or abnormal records are carried out through the audit verification.
7. The method of claim 6, wherein: the managing and updating the original metadata and the metadata to be updated, and the processing result of the metadata comprises:
comparing the metadata to be updated with the original metadata, and updating the original metadata according to a comparison result to obtain updated metadata;
classifying the updated metadata according to the data type to obtain a metadata classification result, wherein the metadata classification result comprises character metadata, date metadata and numerical metadata;
performing field extraction on the updated metadata to obtain a field extraction result, wherein the field extraction result comprises a field name, a field Chinese name and a field type;
judging whether the corresponding metadata accords with the guest group condition or not according to the metadata classification result and the field extraction result, and if so, generating corresponding guest group information;
judging whether the corresponding metadata accords with screening conditions or not according to the metadata classification result and the field extraction result, and if so, generating corresponding screening information;
and integrating the guest group information and the screening information to obtain a metadata processing result.
8. The method of claim 7, wherein: the method for constructing the filtering logic tree according to the metadata processing result and the preset filtering dimensionality, performing recursive operation on the filtering logic tree in a recursive mode to obtain a recursive result expression, and obtaining the target guest group based on the recursive result expression includes the following steps:
generating a guest group rule information table for screening guest groups according to the selection results of the guest group information and the screening information;
generating a preset passenger group screening information table for additionally screening the passenger groups through the predefined passenger groups;
selecting a relation operator for carrying out relation logic operation;
selecting corresponding guest group rule information from the guest group rule information table according to guest group screening requirements, selecting corresponding preset guest group screening information from the preset guest group screening information table, selecting corresponding relation operators from the relation components, and building a screening logic tree based on the rule information, the preset guest group screening information and the relation operators;
and carrying out recursion operation on the screening logic tree to obtain a recursion result expression, and obtaining a target guest group based on the recursion result expression.
9. The method of claim 8, wherein: and the lowest layer of the recursive operation is upwards processed by layer-by-layer relation operation.
10. The method of claim 9, wherein: and constructing a target guest group data test flow according to the target guest group and preset test conditions to obtain a test result, carrying out audit verification on the test result, and judging whether the test is passed or abnormal records are carried out through the audit verification.
Generating a target passenger group information table according to the screened target passenger groups;
generating a test rule table according to a preset test screening rule;
generating a test event table according to a preset test screening event;
selecting corresponding target guest group information from the guest group information table, selecting corresponding test rules from the test rule table, selecting corresponding test events from the test event table, and building a target guest group data test flow for the target guest group information, the test rules and the test events according to test requirements;
and presetting an expected screening value, analyzing and comparing the test result of the target customer group data test flow according to the expected screening value, if the expected screening value is in line with the expected screening value, indicating that the test result is normal, passing the audit verification, and if the expected screening value is not in line with the expected screening value, indicating that the test result is abnormal, interrupting the test and recording to obtain a test result recording table.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211641734.9A CN115858377B (en) | 2022-12-20 | 2022-12-20 | Data testing system and method based on guest group management |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211641734.9A CN115858377B (en) | 2022-12-20 | 2022-12-20 | Data testing system and method based on guest group management |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115858377A true CN115858377A (en) | 2023-03-28 |
| CN115858377B CN115858377B (en) | 2023-09-15 |
Family
ID=85674532
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211641734.9A Active CN115858377B (en) | 2022-12-20 | 2022-12-20 | Data testing system and method based on guest group management |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115858377B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060106856A1 (en) * | 2004-11-04 | 2006-05-18 | International Business Machines Corporation | Method and system for dynamic transform and load of data from a data source defined by metadata into a data store defined by metadata |
| CN101957861A (en) * | 2010-10-18 | 2011-01-26 | 江苏大学 | Novel metadata server cluster and metadata management method based on reconciliation statement |
| CN110008246A (en) * | 2019-02-18 | 2019-07-12 | 新智云数据服务有限公司 | Metadata management method and device |
| CN111858584A (en) * | 2020-06-22 | 2020-10-30 | 中国平安财产保险股份有限公司 | Metadata management method, system and storage medium |
| CN115374329A (en) * | 2022-10-25 | 2022-11-22 | 杭州比智科技有限公司 | Method and system for managing enterprise business metadata and technical metadata |
| CN115617776A (en) * | 2022-09-30 | 2023-01-17 | 国家石油天然气管网集团有限公司 | Data management system and method |
-
2022
- 2022-12-20 CN CN202211641734.9A patent/CN115858377B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060106856A1 (en) * | 2004-11-04 | 2006-05-18 | International Business Machines Corporation | Method and system for dynamic transform and load of data from a data source defined by metadata into a data store defined by metadata |
| CN101957861A (en) * | 2010-10-18 | 2011-01-26 | 江苏大学 | Novel metadata server cluster and metadata management method based on reconciliation statement |
| CN110008246A (en) * | 2019-02-18 | 2019-07-12 | 新智云数据服务有限公司 | Metadata management method and device |
| CN111858584A (en) * | 2020-06-22 | 2020-10-30 | 中国平安财产保险股份有限公司 | Metadata management method, system and storage medium |
| CN115617776A (en) * | 2022-09-30 | 2023-01-17 | 国家石油天然气管网集团有限公司 | Data management system and method |
| CN115374329A (en) * | 2022-10-25 | 2022-11-22 | 杭州比智科技有限公司 | Method and system for managing enterprise business metadata and technical metadata |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115858377B (en) | 2023-09-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109213754B (en) | Data processing system and data processing method | |
| CN112000849A (en) | Unified label library management method, device, equipment and storage medium | |
| CA2751383A1 (en) | Creation of a data store | |
| CN110377524B (en) | Method, device, system and medium for testing report system | |
| CN106528511A (en) | Form analysis method and device | |
| CN112925664A (en) | Target user determination method and device, electronic equipment and storage medium | |
| CN115964272A (en) | Transaction data automatic testing method, device, equipment and readable storage medium | |
| CN107862067B (en) | Screening method and device for bank loan data query | |
| CN110728452B (en) | System and method for realizing multi-dimensional organization integrated personnel selection control in distributed flow system | |
| CN112732242A (en) | Wide table processing script generation method and device | |
| CN116991955A (en) | Data processing method, device, electronic equipment and computer storage medium | |
| CN115858377B (en) | Data testing system and method based on guest group management | |
| CN116089490A (en) | Data analysis method, device, terminal and storage medium | |
| US20140372386A1 (en) | Detecting wasteful data collection | |
| CN114911511A (en) | Software technology state management system | |
| CN113868141A (en) | Data testing method and device, electronic equipment and storage medium | |
| CN112783758A (en) | Test case library and feature library generation method, device and storage medium | |
| US20160307207A1 (en) | Analytical Functionality Selecting Relevant Market Research Data for Global Reporting | |
| CN116152000B (en) | Method and device for tax data processing | |
| US10884599B2 (en) | Graphical user interface for refocusing the scope of a series of queries | |
| CN109522340B (en) | Data statistical method, device and equipment | |
| Chatzistefanou | Data Warehousing in Business Intelligence and ETL Processes | |
| CN116204577A (en) | Data export method and device, computer equipment and computer readable storage medium | |
| CN117170631A (en) | Metadata-based code-free data chart generation method and system | |
| CN117743730A (en) | Dimension tree-based automatic index association calculation method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |