CN114996319B - Data processing method, device and equipment based on rule engine and storage medium - Google Patents
Data processing method, device and equipment based on rule engine and storage medium Download PDFInfo
- Publication number
- CN114996319B CN114996319B CN202210913579.5A CN202210913579A CN114996319B CN 114996319 B CN114996319 B CN 114996319B CN 202210913579 A CN202210913579 A CN 202210913579A CN 114996319 B CN114996319 B CN 114996319B
- Authority
- CN
- China
- Prior art keywords
- rule
- data
- metadata
- preset
- entity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24564—Applying rules; Deductive queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24554—Unary operations; Data partitioning operations
- G06F16/24556—Aggregation; Duplicate elimination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
- G06F16/24573—Query processing with adaptation to user needs using data annotations, e.g. user-defined metadata
Abstract
The application discloses a data processing method, a data processing device, data processing equipment and a storage medium based on a rule engine, and relates to the technical field of computer software development. The method comprises the following steps: acquiring metadata and preset business rules determined after different data sources are filled in a uniform data filling format; the metadata are used for recording an ontology related to the preset business rule and determining corresponding derivative indexes by utilizing a target ontology in the metadata in an aggregation manner; the target ontology is an ontology selected from the metadata according to user requirements; acquiring related data corresponding to the metadata in a preset data acquisition mode, and loading the related data into a corresponding entity; and performing batch rule operation on the entity according to the preset service rule so as to return a corresponding processing result. Through the technical scheme, the rule operation efficiency can be improved, and the application range of the rule engine is widened.
Description
Technical Field
The invention relates to the technical field of computer software development, in particular to a data processing method, a data processing device, data processing equipment and a storage medium based on a rule engine.
Background
At present, a great number of rule engines are available to implement the separation of business rules from business application program codes, and can write rules using semantic modules predefined by the rule engines, accept data input and execute according to the rules, and return the results of the rule execution. However, the currently existing rules engines have the following disadvantages: (1) Lack of interpretability makes the results from this ineffectively applicable; the capability of rule completion is lacked, and special service scenes such as tax benefits and the like cannot be met. (2) The traditional rule engine does not provide a standard for the access data, so that the semantics of the rule engine need to compile rules of different versions in different environments, and the access cost, the maintenance cost and the judgment accuracy cost are high. (3) Distributed operation is limited to rule calculation optimization under the condition of large optimization intervention quantity, and the time consumed after the engine rule is increased is linearly increased, so that the time consumption is long, and the operation efficiency is low.
In summary, how to provide interpretability, improve data access efficiency, and improve rule operation efficiency for a service application scenario is a problem to be solved at present.
Disclosure of Invention
In view of the above, the present invention provides a data processing method, apparatus, device and storage medium based on a rule engine, which can provide interpretability, improve data access efficiency and improve rule operation efficiency for a service application scenario. The specific scheme is as follows:
in a first aspect, the present application discloses a data processing method based on a rule engine, including:
acquiring metadata and preset business rules determined after different data sources are filled in a uniform data filling format; the metadata are used for recording an ontology related to the preset business rule and determining a corresponding derivative index by aggregating a target ontology in the metadata; the target ontology is an ontology selected from the metadata according to user requirements;
acquiring related data corresponding to the metadata in a preset data acquisition mode, and loading the related data into a corresponding entity;
and performing batch rule operation on the entity according to the preset service rule so as to return a corresponding processing result.
Optionally, after obtaining the metadata and the preset business rule determined after the different data sources are filled in the uniform data filling format, the method further includes:
disassembling the preset business rule to obtain different rule sentences;
and configuring the rule statement into a corresponding rule grammar so as to carry out grammar check and grammar test on the rule grammar.
Optionally, after obtaining the metadata and the preset business rule determined after the different data sources are filled in the uniform data filling format, the method further includes:
mapping the ontology to fields in different data sources, and generating different configuration items corresponding to the fields;
and performing data packing on the metadata according to the configuration items to obtain an instance corresponding to the ontology.
Optionally, the obtaining, by a preset data obtaining manner, related data corresponding to the metadata, and loading the related data into a corresponding entity includes:
and acquiring the instance in a preset data acquisition mode, and automatically executing corresponding access logic according to the configuration item so as to load the instance and the body corresponding to the instance as corresponding entities.
Optionally, in the process of acquiring the related data corresponding to the metadata in a preset data acquisition manner and loading the related data into the corresponding entity, the method further includes:
and loading the preset business rules, and performing matching judgment on the bodies in different entities one by using the target rules obtained after loading so as to perform batch processing on the entities.
Optionally, the step of performing matching judgment on the entities in the different entities one by using the target rule obtained after loading includes:
if the matching judgment of the current body and the target rule is completed, placing a zone bit on the current body, and continuing to use the target rule to perform matching judgment with other bodies;
and after all the ontology matching judgment is completed, merging the instances which have the same instance and correspond to different target rules, and then pre-compiling the merged target rules in the instances.
Optionally, the performing batch rule operation on the entity according to the preset service rule to return a corresponding processing result includes:
aligning the body appearing in the preset business rule with the body in the entity, and judging whether the entity meets the preset business rule or not;
if the entity meets the preset service rule, performing batch rule operation on the entity according to the preset service rule to return a corresponding processing result;
if the entity does not meet the preset service rule, outputting a data item which does not meet the preset service rule in the entity, and providing a completion suggestion for enabling the entity to meet the preset service rule for the data item.
In a second aspect, the present application discloses a data processing apparatus based on a rule engine, comprising:
the data acquisition module is used for acquiring metadata and preset business rules determined after different data sources are filled in a uniform data filling format; the metadata are used for recording an ontology related to the preset business rule and determining corresponding derivative indexes by utilizing a target ontology in the metadata in an aggregation manner; the target ontology is an ontology selected from the metadata according to user requirements;
the entity loading module is used for acquiring related data corresponding to the metadata in a preset data acquisition mode and loading the related data into a corresponding entity;
and the rule operation module is used for performing batch rule operation on the entity according to the preset service rule so as to return a corresponding processing result.
In a third aspect, the present application discloses an electronic device comprising a processor and a memory; wherein the memory is used for storing a computer program which is loaded and executed by the processor to implement the rule engine based data processing method as described above.
In a fourth aspect, the present application discloses a computer readable storage medium for storing a computer program; wherein the computer program when executed by a processor implements a rules engine based data processing method as described above.
In the application, firstly, different data sources are obtained and filled in a uniform data filling format, and then metadata and preset service rules are determined; the metadata are used for recording an ontology related to the preset business rule and determining a corresponding derivative index by aggregating a target ontology in the metadata; the target ontology is an ontology selected from the metadata according to user requirements; then, acquiring related data corresponding to the metadata in a preset data acquisition mode, and loading the related data into a corresponding entity; and finally, performing batch rule operation on the entity according to the preset service rule so as to return a corresponding processing result. Therefore, the data access format of the rule engine is unified, relevant data corresponding to the metadata are obtained and loaded as corresponding data entities to be used as data access, and the conventional data input mode facing to the data table is abandoned. The uniformity of data access is guaranteed, different running environments such as online, offline and near-line can be executed by using one set of rule codes, only one set of rule codes needs to be maintained in different environments, different environment data sources can be mixed and used, and the application range of the rule engine is greatly enlarged. And performing batch rule operation on the entity according to the preset service rule, thereby realizing intensive batch rule calculation and greatly improving the operation efficiency of the service rule. In addition, according to the technical scheme of the application, each data item in each entity is loaded into the memory when the rules are executed, and all derived indexes are processed by the attributes in the ontology, so that which data item of each business rule does not reach the standard and can be directly output, all derived indexes can be quickly traced, the data source of the ontology is provided, the separation of the rules and the results is solved, and the interpretability is provided for the business application scene.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flow chart of a method for rule engine based data processing as disclosed herein;
FIG. 2 is a schematic diagram of a data flow of a rules engine based data processing system as disclosed herein;
FIG. 3 is a diagram illustrating exemplary tax domain model data processing according to the present disclosure;
FIG. 4 is a flow chart of a specific rule engine based data processing method disclosed herein;
FIG. 5 is a schematic diagram of a rules engine based data processing system according to the present disclosure;
FIG. 6 is a schematic diagram of a data processing apparatus based on a rules engine according to the present disclosure;
fig. 7 is a block diagram of an electronic device disclosed in the present application.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The existing rule engine has the following defects: (1) Lack of interpretability makes the results from this ineffectively applicable; the capability of rule completion is lacked, and special service scenes such as tax benefits and the like cannot be met. (2) The traditional rule engine does not provide a standard for the access data, so that the semantics of the rule engine need to compile rules of different versions in different environments, and the access cost, the maintenance cost and the judgment accuracy cost are high. (3) Distributed operation is limited to rule calculation optimization under the condition of large optimization intervention quantity, and the time consumed after the engine rule is increased is linearly increased, so that the time consumption is long, and the operation efficiency is low.
Therefore, the data processing scheme based on the rule engine is provided, and the interpretability, the data access efficiency and the rule operation efficiency can be improved for a business application scene.
The embodiment of the invention discloses data processing based on a rule engine, and as shown in figure 1, the method comprises the following steps:
step S11: and acquiring metadata and preset business rules determined after different data sources are filled in a uniform data filling format.
Because the traditional rule engines are all data input modes oriented to the data table, no specification is provided for the accessed data, and the semantics of the rule engines need to write different versions of rules in different environments. In the embodiment of the application, the metadata and the preset service rule are determined by facing different data sources and filling the different data sources in a uniform data filling format, so that the data of the metadata definition layer and the preset service rule definition layer are obtained, and further, the related data can be pulled according to the metadata and the preset service rule. The supported data sources comprise a data warehouse, a MYSQL service library, an online interface and the like.
In the embodiment of the application, the metadata is used for recording an ontology related to the preset business rule and determining a corresponding derivative index by aggregating a target ontology in the metadata; the target ontology is an ontology selected from the metadata according to user requirements. For example, if business data related to the tax field needs to be processed currently, the metadata definition layer defines related ontologies and derived indexes according to actual tax rules. For example, defining the fact elements of taxpayer type, taxpayer industry, all invoice set of taxpayer, etc. in the taxpayer body; the invoice body contains the fact elements of taxpayer identification numbers of both sales parties, invoice types, the industry of the sales parties, whether the invoice is a sales invoice and the like.
It can be understood that an ontology is a well-known conceptual framework, and in the embodiment of the present application, a policy rule ontology is an attribute included in a policy itself; the invoice body is a related element set with the invoice granularity; the taxpayer body is the taxpayer related information such as file number, enterprise area information and the like, and the invoice body can be assembled and nested into the taxpayer body.
Further, after the metadata definition of the ontology is completed, metadata of the derived index can be processed and aggregated for the target ontology in the ontology. The derived indexes are data indexes generated by processing and polymerizing the ontology fact elements, such as: indexes such as growth rate, average value and the like are assembled by fact elements and are derived indexes which are multiplexed in a plurality of rules, so that all derived indexes can be traced quickly, and the data source of the most source of the ontology is provided. It should be noted that the derivation index is a derivation index processed by aggregation according to the defined attributes in the ontology, and is essentially a calculation logic and rule, and the calculation rule is given when the metadata of the derivation index is defined, but specific data is not given.
In the embodiment of the application, after the preset service rule is obtained, the preset service rule is disassembled to obtain different rule statements, that is, the preset service rule is disassembled into specific rules according to the actual preset service rule. The rule statements are then configured into corresponding rule grammars for syntax checking and syntax testing of the rule grammars, it being understood that only rules that pass syntax checking and syntax testing can be used and that rules that pass syntax checking and syntax testing are stored for subsequent invocation.
Step S12: and acquiring related data corresponding to the metadata in a preset data acquisition mode, and loading the related data into a corresponding entity.
In the embodiment of the application, after the metadata and the preset business rule are obtained, data filling is carried out according to the ontology in the metadata and the preset business rule. Specifically, the ontology is mapped to fields in different data sources, and different configuration items corresponding to the fields are generated; and performing data packing on the metadata according to the configuration items to obtain an instance corresponding to the ontology. For example, the ontology in the obtained metadata is mapped to fields in an actual database, fields in an interface and the like, and different configuration items are formed to provide the entity access JAVA package, the interface address and the mapping table metadata to the rule engine when the rule engine is accessed in cooperation with different environments, so that the rule engine can obtain ontology instance data in the calculation process.
In the embodiment of the application, the instance is acquired by a preset data acquisition mode, for example, relevant data is acquired by an interface/Map Reduce (programming model) task/JDBC (Java DataBase Connectivity, java DataBase connection) mode; it can be understood that the acquired related data are data such as a corresponding ontology, ontology instance data filled with the data, and corresponding derivative indexes, and then corresponding access logic is automatically executed according to configuration items formed by different data sources, so as to load the ontology corresponding to the instance and the instance into corresponding entities. That is, the entity refers to a combination of an ontology and an instance, for example, a policy rule, an invoice data, and a taxpayer data are all entities.
In the embodiment of the application, after the related data of the metadata is acquired, the rule engine loads the data. When the rules are loaded, in order to improve the judgment efficiency of the preset business rules on the entities, the loaded target rules are used for carrying out matching judgment on the bodies in different entities one by one so as to carry out batch processing on the entities.
If the matching judgment of the current body and the target rule is finished, placing a zone bit on the current body, and continuing to perform matching judgment with other bodies by using the target rule; and after all the ontology matching judgment is completed, merging the instances which have the same instance and correspond to different target rules, and then pre-compiling the merged target rules in the instances.
Step S13: and performing batch rule operation on the entity according to the preset service rule so as to return a corresponding processing result.
In the embodiment of the application, each loaded entity executes the preset service rule, performs batch rule operation, and then provides data in various forms, such as an interface, a data warehouse, a service database and the like, for the data according to the requirements of a user according to the calculation result.
In the embodiment of the application, the rules are processed in batch when the rules are loaded, so that multiple pieces of rule running test repeated loading data can be changed into single instance unified loading data and batch running test rules when the rules are calculated. Therefore, only one time of loading example data is needed for 1 run of 10 10000 rules, and the original linear time complexity is reduced to a constant level. The performance of one example under the condition of multiple tax policies and multiple tax benefits is greatly improved under the condition that multiple policy checks need to be operated.
Fig. 2 shows the data flow of the whole system when data processing is performed. The rule engine is used for: presetting a service rule, a body and derived indexes needed by the rule, pulling related data to load into a corresponding entity in the modes of an interface/Map Reduce task/JDBC and the like according to a configuration item corresponding to an access data source, and performing corresponding rule calculation for each instance. The data sources can be data asset platforms such as an online production service library/an online system interface/a data warehouse, and different data sources are filled in a uniform data filling format, so that the access cost is reduced. And finally, returning the corresponding calculation result to the calling party, the corresponding data warehouse and the corresponding service library by the rule engine so as to support (online, offline and online) running test.
For example, as shown in fig. 3, when the embodiment of the present application is applied to a tax field model, a taxpayer body, an invoice body, and a tax policy body are vertically established. The entity model management platform is responsible for predefining an ontology and complex derivative index calculation logics and rules in the whole link (namely, the derivative indexes processed by aggregation according to the defined attributes in the ontology are essentially calculation logics and rules); tax model management platform: the system is responsible for carrying out rule configuration by utilizing the grammar predefined by the rule engine according to the attribute set of the ontology; a rule engine: providing a distributed rule execution function; providing a data entity loading function; providing a rule completion function; providing a rule explanatory function; data loading layer: and filling data for the entity model management platform according to the ontology defined by the entity model management platform and the standard specification of the rule engine (providing an interface and a database on line; providing a JAR packet for providing access of the entity off line).
According to the method, firstly, metadata and preset business rules determined after different data sources are filled in a uniform data filling format are obtained; the metadata are used for recording an ontology related to the preset business rule and determining a corresponding derivative index by aggregating a target ontology in the metadata; the target ontology is selected from the metadata according to user requirements; then, acquiring related data corresponding to the metadata in a preset data acquisition mode, and loading the related data into a corresponding entity; and finally, performing batch rule operation on the entity according to the preset service rule so as to return a corresponding processing result. Therefore, the data access format of the rule engine is unified, relevant data corresponding to the metadata are obtained and loaded as corresponding data entities to be used as data access, and the conventional data input mode facing to the data table is abandoned. The uniformity of data access is guaranteed, different running environments such as online, offline and near-line can be executed by using one set of rule codes, only one set of rule codes needs to be maintained in different environments, different environment data sources can be mixed and used even, and the application range of the rule engine is greatly enlarged. And performing batch rule operation on the entity according to the preset service rule, thereby realizing intensive batch rule calculation and greatly improving the operation efficiency of the service rule. In addition, according to the technical scheme of the application, each data item in each entity is loaded into the memory when the rules are executed, and all derived indexes are processed by the attributes in the ontology, so that which data item of each business rule does not reach the standard and can be directly output, all derived indexes can be quickly traced, the data source of the ontology is provided, the separation of the rules and the results is solved, and the interpretability is provided for the business application scene.
The embodiment of the application discloses a specific data processing method based on a rule engine, and as shown in fig. 4, the method includes:
step S21: and obtaining metadata and preset business rules determined after different data sources are filled in a uniform data filling format.
Step S22: and acquiring related data corresponding to the metadata in a preset data acquisition mode, and loading the related data into a corresponding entity.
For more specific processing procedures of the step S21 and the step S22, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
Step S23: and loading the preset business rules, and performing matching judgment on the bodies in different entities one by using the target rules obtained after loading so as to perform batch processing on the entities.
In the embodiment of the application, after the related data of the metadata is acquired, the rule engine loads the data. When the rules are loaded, in order to improve the judgment efficiency of the preset business rules on the entities, the loaded target rules are used for carrying out matching judgment on the bodies in different entities one by one so as to carry out batch processing on the entities. For example, if the taxpayer is Zhang III, li IV and Wang V, the body conditions to be judged are taxpayer type, the industry to which the taxpayer belongs and tax-containing sales income, if the taxpayer is determined to belong to an individual industrial and commercial business, the industry to which the taxpayer belongs is the service industry and the taxpayer with tax-containing sales income of 2 ten thousand yuan, the body is judged one by one, if the taxpayer type is judged first, the taxpayer meeting the conditions is screened out, then the industry to which the taxpayer belongs is judged, and finally the tax-containing sales income is judged.
Step S24: and if the matching judgment of the current body and the target rule is completed, placing a zone bit on the current body, and continuously utilizing the target rule to perform matching judgment with other bodies.
It should be noted that, if the matching judgment between the current body and the target rule is completed, a flag bit is placed in the current body, and the target rule is continuously utilized to perform matching judgment with other bodies. For example, if one is three for a taxpayer, there are two pieces of data: one including three taxpayer types and affiliated industries, and the other including three affiliated industries and income including tax sales, merging the data related to the three taxpayers to obtain one including three taxpayer types, affiliated industries and income including tax sales.
Step S25: and after all the ontology matching judgment is completed, merging the instances which have the same instance and correspond to different target rules, and then pre-compiling the merged target rules in the instances.
In the embodiment of the application, the operation efficiency of the rules can be improved by merging the corresponding target rules with the same examples, and then the operation speed of the rules can be accelerated by pre-compiling the merged rules.
Step S26: and aligning the body appearing in the preset business rule with the body in the entity, and judging whether the entity meets the preset business rule.
In the embodiment of the present application, a batch of rule operations are performed on each loaded entity. Specifically, firstly, a preloaded rule, namely a precompiled target rule, is executed in batch for a loaded entity; subsequently, unlike the past methods, alignment and matching of instances, rules, policies and suggestions based on semantic embedding are performed simultaneously in the rule calculation process, thereby implementing the rule operation of the entity.
Step S27: and if the entity meets the preset service rule, performing batch rule operation on the entity according to the preset service rule to return a corresponding processing result.
When entity judgment is carried out, if the entity meets preset business rules, for example, the taxpayer type with taxpayer three belongs to an individual industrial and commercial customer, the industry belongs to the service industry, and the tax-containing sales income is 2 ten thousand yuan, and when the rules are judged, the taxpayer type belonging to the individual industrial and commercial customer, the industry belongs to the service industry, and the tax-containing sales income is not less than 1 ten thousand yuan is output, the three-part meeting conditions are obtained, corresponding data support is obtained, and data output is carried out on the taxpayer.
Step S28: if the entity does not meet the preset service rule, outputting a data item which does not meet the preset service rule in the entity, and providing a completion suggestion for enabling the entity to meet the preset service rule for the data item.
In the embodiment of the application, after the rule is executed, the reason why the condition is met or not met is deduced according to the relation between the input data and the formalized rule, wherein the reason includes source policies, preferential rules and the like of business rules. If the entity does not satisfy the preset business rule, for example, in a specific implementation mode, data of Zhang III of taxpayers exist, the data comprises that the taxpayers belong to individual industrial and commercial businesses and belong to the business, tax-containing sales amount of the taxpayers is absent when the rule judgment is carried out, the data corresponding to Zhang III belongs to the data lacking the preset business rule judgment condition, and a completion suggestion that the tax-containing sales amount needs to be completed is put forward for the data of Zhang III. In another specific implementation mode, data of taxpayer three-in-one exists, the data comprises that the taxpayer type belongs to an individual industrial business, the industry belongs to a service industry, and the tax-containing sales income is 8000 yuan, if the taxpayer type belongs to the individual industrial business, the industry belongs to the service industry, and the tax-containing sales income is not less than 1 ten thousand yuan, three-in-one does not meet the condition, and the income is less than the income specified by the rule, the suggestion of sales promotion is given to pay the tax.
According to the method, firstly, metadata and preset business rules determined after different data sources are filled in a uniform data filling format are obtained; the metadata are used for recording an ontology related to the preset business rule and determining a corresponding derivative index by aggregating a target ontology in the metadata; the target ontology is selected from the metadata according to user requirements; then, acquiring related data corresponding to the metadata in a preset data acquisition mode, and loading the related data into a corresponding entity; loading the preset business rules, and performing matching judgment on the bodies in different entities one by using the target rules obtained after loading so as to perform batch processing on the entities; if the matching judgment of the current body and the target rule is finished, placing a zone bit on the current body, and continuing to use the target rule to perform matching judgment with other bodies; after all the ontology matching judgment is completed, merging the instances which have the same instance and correspond to different target rules, and then pre-compiling the merged target rules in the instances; aligning the body appearing in the preset business rule with the body in the entity, and judging whether the entity meets the preset business rule or not; and if the entity meets the preset service rule, performing batch rule operation on the entity according to the preset service rule to return a corresponding processing result. If the entity does not meet the preset service rule, outputting a data item which does not meet the preset service rule in the entity, and providing a completion suggestion for enabling the entity to meet the preset service rule for the data item. Therefore, the data access format of the rule engine is unified, relevant data corresponding to the metadata are obtained and loaded into corresponding data entities to serve as data access, and the conventional data input mode facing to a data table is abandoned. The uniformity of data access is guaranteed, different running environments such as online, offline and near-line can be executed by using one set of rule codes, only one set of rule codes needs to be maintained in different environments, different environment data sources can be mixed and used even, and the application range of the rule engine is greatly enlarged. And performing batch rule operation on the entity according to the preset service rule, thereby realizing intensive batch rule calculation and greatly improving the operation efficiency of the service rule. In addition, according to the technical scheme of the application, each data item in each entity is loaded into the memory when the rules are executed, and all derived indexes are processed by the attributes in the ontology, so that which data item of each business rule does not reach the standard and can be directly output, all derived indexes can be quickly traced, the data source of the ontology is provided, the separation of the rules and the results is solved, and the interpretability is provided for the business application scene.
Fig. 5 is a schematic diagram of a specific system for data processing based on a rule engine. 1. The metadata definition layer is responsible for defining the ontology and the derived indexes. For example, defining the fact elements of taxpayer type, taxpayer industry, all invoice set of taxpayer, etc. in the taxpayer body; defining fact elements such as taxpayer identification numbers of both sales parties, invoice types, sales party industries, whether the invoice is a sales invoice and the like in an invoice body; the derived index defines how to process and aggregate the elements of the ontology, gives the calculation rule but does not give specific data, such as X bits before the invoice tax classification code. 2. The rule definition layer is responsible for decomposing into specific rules according to actual tax rules, configuring the rules into grammars predefined by the rule engine, transmitting the rules to the rule engine for grammar check and grammar test, and storing the rules to the rule engine after the rules pass the grammar check and grammar test. 3. The data loading layer is responsible for mapping the ontology in the metadata definition layer into fields in an actual database and fields in an interface, and forming different configuration items so as to provide the entity access JAVA program package, the interface address and mapping table metadata to the rule engine when the rule engine is accessed in cooperation with different environments, so that the rule engine can acquire ontology instance data in the calculation process. 4. And (1) rule loading in the rule engine is responsible for three steps of rule loading, rule condition merging with an instance and rule precompilation. Firstly, expressing and loading a rule in a structured mode into a system; then, carrying out batch processing on the judgment of the example rule, carrying out one-time judgment on the same condition, placing a flag bit, and combining and disambiguating the same example rule to improve the operation efficiency of the rule; and finally, pre-compiling the rule to accelerate the rule running speed. (2) The data loading in the rule engine is used for performing structured representation and reading on data of different data sources. After the module takes the data defined by other layers, different access logics are executed according to different configuration items, and batch loading of instance data is achieved. Meanwhile, when the data is loaded, the module automatically carries out different data access operations on different data sources, and the supported data sources comprise a data warehouse, a MYSQL service library, an online interface and the like. (3) And the rule operation module in the rule engine performs batch rule operation on each loaded entity. Firstly, executing pre-loaded rules in batch aiming at loaded entities; and then, aligning and matching the examples, the rules, the policies and the suggestions based on semantic embedding are carried out simultaneously in the rule calculation process, so that corresponding data support and completion suggestions are obtained based on the examples and the operation rules, data support is output according to the policies, and completion suggestions are given to the policies which are not according to the policies. (4) And the result returning module in the rule engine provides data in various forms (interfaces, data bins and a business database) for the result calculated by the rule engine according to the requirement of a user. (5) The rule engine also comprises other functional item modules which are owned by the traditional rule engine and comprise platform functions of rule grammar check, rule grammar test, rule grammar storage, rule timing tasks, rule execution state monitoring and the like, and the basic service use of the rule engine is met.
It should be noted that, the data loading position may be changed, and the entity definition layer and the data loading layer only provide the definition of the ontology, the processing mode of the derivative rule, and the mapping relationship between the ontology element and the actual data table database interface for guaranteeing the decoupling and the operation performance between the systems, and are not in charge of data fetching. In addition, the rule engine directly acquires the data table, does not define the entity definition layer and the data loading layer, and essentially acquires the data of the data table, but the rule engine can quickly access different service scenes by using different configurations, so that the service access efficiency is improved, and the rule grammars of the different scenes are unified.
Correspondingly, an embodiment of the present application further discloses a data processing apparatus based on a rule engine, and as shown in fig. 6, the apparatus includes:
the data acquisition module 11 is configured to acquire metadata and preset service rules determined after different data sources are filled in a uniform data filling format; the metadata are used for recording an ontology related to the preset business rule and determining a corresponding derivative index by aggregating a target ontology in the metadata; the target ontology is an ontology selected from the metadata according to user requirements;
an entity loading module 12, configured to obtain, in a preset data obtaining manner, related data corresponding to the metadata, and load the related data as a corresponding entity;
and the rule operation module 13 is configured to perform batch rule operation on the entity according to the preset service rule so as to return a corresponding processing result.
For more specific working processes of the modules, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
Therefore, according to the scheme of the embodiment, firstly, the metadata and the preset business rules determined after different data sources are filled in a uniform data filling format are obtained; the metadata are used for recording an ontology related to the preset business rule and determining a corresponding derivative index by aggregating a target ontology in the metadata; the target ontology is an ontology selected from the metadata according to user requirements; then, acquiring related data corresponding to the metadata in a preset data acquisition mode, and loading the related data into a corresponding entity; and finally, performing batch rule operation on the entity according to the preset service rule so as to return a corresponding processing result. Therefore, the data access format of the rule engine is unified, relevant data corresponding to the metadata are obtained and loaded into corresponding data entities to serve as data access, and the conventional data input mode facing to a data table is abandoned. The uniformity of data access is guaranteed, different running environments such as online, offline and near-line can be executed by using one set of rule codes, only one set of rule codes needs to be maintained in different environments, different environment data sources can be mixed and used even, and the application range of the rule engine is greatly enlarged. And performing batch rule operation on the entities according to the preset service rules, thereby realizing intensive batch rule calculation and greatly improving the operation efficiency of the service rules. In addition, according to the technical scheme of the application, each data item in each entity is loaded into the memory when the rules are executed, and all derived indexes are processed by the attributes in the ontology, so that which data item of each business rule does not reach the standard and can be directly output, all derived indexes can be quickly traced, the data source of the ontology is provided, the separation of the rules and the results is solved, and the interpretability is provided for the business application scene.
Further, an electronic device is disclosed in the embodiments of the present application, and fig. 7 is a block diagram of an electronic device 20 according to an exemplary embodiment, which should not be construed as limiting the scope of the application.
Fig. 7 is a schematic structural diagram of an electronic device 20 according to an embodiment of the present disclosure. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input output interface 25, and a communication bus 26. Wherein, the memory 22 is used for storing a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the rule engine-based data processing method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in the present embodiment may be a computer.
In this embodiment, the power supply 23 is configured to provide a working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and an external device, and a communication protocol followed by the communication interface is any communication protocol applicable to the technical solution of the present application, and is not specifically limited herein; the input/output interface 25 is configured to obtain external input data or output data to the outside, and a specific interface type thereof may be selected according to specific application requirements, which is not specifically limited herein.
In addition, the memory 22 is used as a carrier for storing resources, and may be a read-only memory, a random access memory, a magnetic disk, an optical disk, or the like, the resources stored thereon may include an operating system 221, a computer program 222, data 223, and the like, and the data 223 may include various data. The storage means may be a transient storage or a permanent storage.
The operating system 221 is used for managing and controlling each hardware device on the electronic device 20 and the computer program 222, and may be Windows Server, netware, unix, linux, or the like. The computer program 222 may further include a computer program that can be used to perform other specific tasks in addition to the computer program that can be used to perform the rule engine-based data processing method disclosed in any of the foregoing embodiments and executed by the electronic device 20.
Further, embodiments of the present application disclose a computer-readable storage medium, where the computer-readable storage medium includes a Random Access Memory (RAM), a Memory, a Read-Only Memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a magnetic disk, or an optical disk or any other form of storage medium known in the art. Wherein the computer program, when executed by a processor, implements the aforementioned rule engine-based data processing method. For the specific steps of the method, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
The steps of a rules engine-based data process or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising one of 8230; \8230;" 8230; "does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.
The rule engine-based data processing method, apparatus, device and storage medium provided by the present invention are described in detail above, and a specific example is applied in the present document to explain the principle and implementation of the present invention, and the description of the above embodiment is only used to help understanding the method and core ideas of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (8)
1. A data processing method based on a rule engine is characterized by comprising the following steps:
acquiring metadata and a preset service rule which are determined after different data sources are filled in a uniform data filling format; the metadata are used for recording an ontology related to the preset business rule and determining a corresponding derivative index by aggregating a target ontology in the metadata; the target ontology is an ontology selected from the metadata according to user requirements;
acquiring related data corresponding to the metadata in a preset data acquisition mode, and loading the related data into a corresponding entity;
performing batch rule operation on the entity according to the preset service rule to return a corresponding processing result;
the process of acquiring the related data corresponding to the metadata in a preset data acquisition mode and loading the related data into the corresponding entity further comprises:
loading the preset business rules, and performing matching judgment on the bodies in different entities one by using the target rules obtained after loading so as to perform batch processing on the entities;
the method for carrying out matching judgment on the bodies in the different entities one by utilizing the target rule obtained after loading comprises the following steps:
if the matching judgment of the current body and the target rule is finished, placing a zone bit on the current body, and continuing to use the target rule to perform matching judgment with other bodies; and after all the ontology matching judgment is completed, merging the instances which have the same instance and correspond to different target rules, and then pre-compiling the merged target rules in the instances.
2. The data processing method based on the rule engine as claimed in claim 1, wherein after obtaining the metadata and the preset business rules determined after the different data sources are packed in the uniform data packing format, the method further comprises:
disassembling the preset business rule to obtain different rule sentences;
and configuring the rule statement into a corresponding rule grammar so as to carry out grammar check and grammar test on the rule grammar.
3. The data processing method based on the rule engine as claimed in claim 1, wherein after obtaining the metadata and the preset business rules determined after the different data sources are packed in the uniform data packing format, the method further comprises:
mapping the ontology to fields in different data sources, and generating different configuration items corresponding to the fields;
and performing data packing on the metadata according to the configuration items to obtain an instance corresponding to the ontology.
4. The data processing method based on the rule engine as claimed in claim 3, wherein the obtaining of the related data corresponding to the metadata by a preset data obtaining manner and the loading of the related data into the corresponding entity comprises:
and acquiring the examples in a preset data acquisition mode, and automatically executing corresponding access logic according to the configuration items so as to load the examples and the bodies corresponding to the examples into corresponding entities.
5. The rules engine-based data processing method of any of claims 1 to 4, wherein the performing batch rule operations on the entities according to the preset business rules to return corresponding processing results comprises:
aligning the body appearing in the preset business rule with the body in the entity, and judging whether the entity meets the preset business rule or not;
if the entity meets the preset service rule, performing batch rule operation on the entity according to the preset service rule to return a corresponding processing result;
if the entity does not meet the preset service rule, outputting a data item which does not meet the preset service rule in the entity, and providing a completion suggestion for enabling the entity to meet the preset service rule for the data item.
6. A rules engine based data processing apparatus, comprising:
the data acquisition module is used for acquiring metadata and preset business rules determined after different data sources are filled in a uniform data filling format; the metadata are used for recording an ontology related to the preset business rule and determining corresponding derivative indexes by utilizing a target ontology in the metadata in an aggregation manner; the target ontology is selected from the metadata according to user requirements;
the entity loading module is used for acquiring related data corresponding to the metadata in a preset data acquisition mode and loading the related data into a corresponding entity;
the rule operation module is used for performing batch rule operation on the entity according to the preset service rule so as to return a corresponding processing result;
the entity loading module is further used for loading the preset service rules and performing matching judgment on the bodies in different entities one by using the target rules obtained after loading so as to perform batch processing on the entities;
the entity loading module is specifically used for placing a zone bit on the current body and continuously utilizing the target rule to perform matching judgment with other bodies if the matching judgment of the current body and the target rule is completed; and after all the ontology matching judgment is completed, merging the instances which have the same instance and correspond to different target rules, and then pre-compiling the merged target rules in the instances.
7. An electronic device, wherein the electronic device comprises a processor and a memory; wherein the memory is for storing a computer program that is loaded and executed by the processor to implement the rules engine based data processing method of any of claims 1 to 5.
8. A computer-readable storage medium for storing a computer program; wherein the computer program when executed by a processor implements a rules engine based data processing method as claimed in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210913579.5A CN114996319B (en) | 2022-08-01 | 2022-08-01 | Data processing method, device and equipment based on rule engine and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210913579.5A CN114996319B (en) | 2022-08-01 | 2022-08-01 | Data processing method, device and equipment based on rule engine and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114996319A CN114996319A (en) | 2022-09-02 |
| CN114996319B true CN114996319B (en) | 2022-11-04 |
Family
ID=83021512
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210913579.5A Active CN114996319B (en) | 2022-08-01 | 2022-08-01 | Data processing method, device and equipment based on rule engine and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114996319B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110297840A (en) * | 2019-05-22 | 2019-10-01 | 平安银行股份有限公司 | Data processing method, device, equipment and the storage medium of rule-based engine |
| CN113360554A (en) * | 2020-03-06 | 2021-09-07 | 深圳法大大网络科技有限公司 | Method and equipment for extracting, converting and loading ETL (extract transform load) data |
| CN114090653A (en) * | 2021-11-17 | 2022-02-25 | 中通服软件科技有限公司 | Resource data statistical method and device, meta-platform equipment and storage medium |
| CN114611486A (en) * | 2022-03-09 | 2022-06-10 | 上海弘玑信息技术有限公司 | Information extraction engine generation method and device and electronic equipment |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8214351B2 (en) * | 2007-04-16 | 2012-07-03 | International Business Machines Corporation | Selecting rules engines for processing abstract rules based on functionality and cost |
| NZ594759A (en) * | 2010-09-03 | 2013-03-28 | Tata Consultancy Services Ltd | A system and method of managing mapping information |
| US8903755B2 (en) * | 2012-06-07 | 2014-12-02 | International Business Machines Corporation | Implementing meta rules on an executable rule engine |
| GB2509539A (en) * | 2013-01-08 | 2014-07-09 | Ibm | Production rule engine |
| US9384575B2 (en) * | 2013-05-09 | 2016-07-05 | Sap Se | Space constrained small format visual analytic labeling |
| CN106296399A (en) * | 2015-06-11 | 2017-01-04 | 交通银行股份有限公司 | The data processing method of business rule formulation and system |
| US10409834B2 (en) * | 2016-07-11 | 2019-09-10 | Al-Elm Information Security Co. | Methods and systems for multi-dynamic data retrieval and data disbursement |
| CN110888672B (en) * | 2019-11-26 | 2023-05-02 | 北京仁科互动网络技术有限公司 | Expression engine implementation method and system based on metadata architecture |
| US11537618B2 (en) * | 2020-03-18 | 2022-12-27 | Microsoft Technology Licensing, Llc | Compliant entity conflation and access |
| CN111737400B (en) * | 2020-06-15 | 2023-06-20 | 上海理想信息产业（集团）有限公司 | Knowledge reasoning-based big data service label expansion method and system |
| CN112948486A (en) * | 2021-02-04 | 2021-06-11 | 北京淇瑀信息科技有限公司 | Batch data synchronization method and system and electronic equipment |
| CN113641705B (en) * | 2021-08-16 | 2024-04-26 | 神州数码融信软件有限公司 | Marketing disposal rule engine method based on calculation engine |
| CN113742493A (en) * | 2021-09-01 | 2021-12-03 | 志诺维思（北京）基因科技有限公司 | Method and device for constructing pathological knowledge map |
| CN114443042A (en) * | 2021-12-13 | 2022-05-06 | 北京国电通网络技术有限公司 | Service arrangement execution method based on rule engine and related equipment |
-
2022
- 2022-08-01 CN CN202210913579.5A patent/CN114996319B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110297840A (en) * | 2019-05-22 | 2019-10-01 | 平安银行股份有限公司 | Data processing method, device, equipment and the storage medium of rule-based engine |
| CN113360554A (en) * | 2020-03-06 | 2021-09-07 | 深圳法大大网络科技有限公司 | Method and equipment for extracting, converting and loading ETL (extract transform load) data |
| CN114090653A (en) * | 2021-11-17 | 2022-02-25 | 中通服软件科技有限公司 | Resource data statistical method and device, meta-platform equipment and storage medium |
| CN114611486A (en) * | 2022-03-09 | 2022-06-10 | 上海弘玑信息技术有限公司 | Information extraction engine generation method and device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114996319A (en) | 2022-09-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8555248B2 (en) | Business object change management using release status codes | |
| US9542469B2 (en) | Data warehouse data model adapters | |
| EP3107050A1 (en) | System for data aggregation and report generation | |
| CN111078702B (en) | SQL sentence classification management and unified query method and device | |
| CN102542382A (en) | Method and device for managing business rule | |
| US9110984B1 (en) | Methods and systems for constructing a taxonomy based on hierarchical clustering | |
| CN109408493A (en) | A kind of moving method and system of data source | |
| CN108540351B (en) | Automatic testing method for distributed big data service | |
| CN110399089B (en) | Data storage method, device, equipment and medium | |
| CN110851513A (en) | Multi-source heterogeneous data reading method and device based on Spark calculation engine | |
| CN112604273B (en) | Data-driven game system function loading method, device and storage medium | |
| CN112256355B (en) | Data-driven system function loading method, equipment and storage medium | |
| CN114996319B (en) | Data processing method, device and equipment based on rule engine and storage medium | |
| WO2022253165A1 (en) | Scheduling method, system, server and computer readable storage medium | |
| CN115115062B (en) | Machine learning model building method, related device and computer program product | |
| US20200320082A1 (en) | Advanced multiprovider optimization | |
| US11645283B2 (en) | Predictive query processing | |
| CN117009397A (en) | Data query method, data query device, electronic equipment and storage medium | |
| CN113688151A (en) | Data access method, device, system, equipment and medium based on virtual database | |
| CN110633459A (en) | Method and system for automatically generating data report and computer readable storage medium | |
| CN111143322A (en) | Data standard treatment system and method | |
| CN113326401B (en) | Method and system for generating field blood relationship | |
| CN113935301B (en) | Report generation method and device, storage medium and electronic equipment | |
| CN114328965A (en) | Knowledge graph updating method and device and computer equipment | |
| CN117648339B (en) | Data exploration method and device, server and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |