CN114996319A - Data processing method, device and equipment based on rule engine and storage medium - Google Patents

Data processing method, device and equipment based on rule engine and storage medium Download PDF

Info

Publication number
CN114996319A
CN114996319A CN202210913579.5A CN202210913579A CN114996319A CN 114996319 A CN114996319 A CN 114996319A CN 202210913579 A CN202210913579 A CN 202210913579A CN 114996319 A CN114996319 A CN 114996319A
Authority
CN
China
Prior art keywords
rule
data
metadata
preset
entity
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210913579.5A
Other languages
Chinese (zh)
Other versions
CN114996319B (en
Inventor
徐煌
张镇潮
施建生
沈懿忱
刘子星
吴华亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Servyou Software Group Co ltd
Original Assignee
Servyou Software Group Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Servyou Software Group Co ltd filed Critical Servyou Software Group Co ltd
Priority to CN202210913579.5A priority Critical patent/CN114996319B/en
Publication of CN114996319A publication Critical patent/CN114996319A/en
Application granted granted Critical
Publication of CN114996319B publication Critical patent/CN114996319B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24564Applying rules; Deductive queries
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24553Query execution of query operations
    • G06F16/24554Unary operations; Data partitioning operations
    • G06F16/24556Aggregation; Duplicate elimination
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2457Query processing with adaptation to user needs
    • G06F16/24573Query processing with adaptation to user needs using data annotations, e.g. user-defined metadata

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Library & Information Science (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application discloses a data processing method, a data processing device, data processing equipment and a storage medium based on a rule engine, and relates to the technical field of computer software development. The method comprises the following steps: acquiring metadata and preset business rules determined after different data sources are filled in a uniform data filling format; the metadata are used for recording an ontology related to the preset business rule and determining a corresponding derivative index by aggregating a target ontology in the metadata; the target ontology is an ontology selected from the metadata according to user requirements; acquiring related data corresponding to the metadata in a preset data acquisition mode, and loading the related data into a corresponding entity; and performing batch rule operation on the entity according to the preset service rule so as to return a corresponding processing result. By the technical scheme, the rule operation efficiency can be improved, and the application range of the rule engine is widened.

Description

Data processing method, device and equipment based on rule engine and storage medium
Technical Field
The invention relates to the technical field of computer software development, in particular to a data processing method, a data processing device, data processing equipment and a storage medium based on a rule engine.
Background
At present, a great number of rule engines are available to implement the separation of business rules from business application program codes, and can write rules using semantic modules predefined by the rule engines, accept data input and execute according to the rules, and return the results of the rule execution. However, the currently existing rules engines have the following disadvantages: (1) lack of interpretability makes the results from this ineffectively applicable; the capability of rule completion is lacked, and special service scenes such as tax benefits and the like cannot be met. (2) The traditional rule engine does not provide a standard for the access data, so that the semantics of the rule engine need to compile rules of different versions in different environments, and the access cost, the maintenance cost and the judgment accuracy cost are high. (3) Distributed operation is limited to rule calculation optimization under the condition of large optimization intervention quantity, and the time consumed after the engine rule is increased is linearly increased, so that the time consumption is long, and the operation efficiency is low.
In summary, how to provide interpretability, improve data access efficiency, and improve rule operation efficiency for a service application scenario is a problem to be solved at present.
Disclosure of Invention
In view of the above, the present invention provides a data processing method, apparatus, device and storage medium based on a rule engine, which can provide interpretability, improve data access efficiency and improve rule operation efficiency for a service application scenario. The specific scheme is as follows:
in a first aspect, the present application discloses a data processing method based on a rule engine, including:
acquiring metadata and a preset service rule which are determined after different data sources are filled in a uniform data filling format; the metadata are used for recording an ontology related to the preset business rule and determining corresponding derivative indexes by utilizing a target ontology in the metadata in an aggregation manner; the target ontology is an ontology selected from the metadata according to user requirements;
acquiring related data corresponding to the metadata in a preset data acquisition mode, and loading the related data into a corresponding entity;
and performing batch rule operation on the entity according to the preset service rule so as to return a corresponding processing result.
Optionally, after obtaining the metadata and the preset business rule determined after the different data sources are filled in the uniform data filling format, the method further includes:
disassembling the preset business rule to obtain different rule sentences;
and configuring the rule statement into a corresponding rule grammar so as to carry out grammar check and grammar test on the rule grammar.
Optionally, after obtaining the metadata and the preset business rule determined after the different data sources are filled in the uniform data filling format, the method further includes:
mapping the ontology to fields in different data sources, and generating different configuration items corresponding to the fields;
and performing data packing on the metadata according to the configuration items to obtain an instance corresponding to the ontology.
Optionally, the obtaining, by a preset data obtaining manner, related data corresponding to the metadata, and loading the related data into a corresponding entity includes:
and acquiring the instance in a preset data acquisition mode, and automatically executing corresponding access logic according to the configuration item so as to load the instance and the body corresponding to the instance as corresponding entities.
Optionally, in the process of acquiring the related data corresponding to the metadata in a preset data acquisition manner and loading the related data into the corresponding entity, the method further includes:
and loading the preset business rules, and performing matching judgment on the bodies in different entities one by using the target rules obtained after loading so as to perform batch processing on the entities.
Optionally, the step of performing matching judgment on the entities in the different entities one by using the target rule obtained after loading includes:
if the matching judgment of the current body and the target rule is finished, placing a zone bit on the current body, and continuing to use the target rule to perform matching judgment with other bodies;
and after all the ontology matching judgment is completed, merging the instances which have the same instance and correspond to different target rules, and then pre-compiling the merged target rules in the instances.
Optionally, the performing batch rule operation on the entity according to the preset service rule to return a corresponding processing result includes:
aligning the body appearing in the preset business rule with the body in the entity, and judging whether the entity meets the preset business rule or not;
if the entity meets the preset service rule, performing batch rule operation on the entity according to the preset service rule to return a corresponding processing result;
if the entity does not meet the preset service rule, outputting a data item which does not meet the preset service rule in the entity, and providing a completion suggestion for enabling the entity to meet the preset service rule for the data item.
In a second aspect, the present application discloses a data processing apparatus based on a rules engine, comprising:
the data acquisition module is used for acquiring metadata and preset business rules determined after different data sources are filled in a uniform data filling format; the metadata are used for recording an ontology related to the preset business rule and determining a corresponding derivative index by aggregating a target ontology in the metadata; the target ontology is selected from the metadata according to user requirements;
the entity loading module is used for acquiring related data corresponding to the metadata in a preset data acquisition mode and loading the related data into a corresponding entity;
and the rule operation module is used for performing batch rule operation on the entity according to the preset service rule so as to return a corresponding processing result.
In a third aspect, the present application discloses an electronic device comprising a processor and a memory; wherein the memory is used for storing a computer program which is loaded and executed by the processor to implement the rule engine based data processing method as described above.
In a fourth aspect, the present application discloses a computer readable storage medium for storing a computer program; wherein the computer program when executed by a processor implements the rules engine based data processing method as described above.
According to the method, firstly, metadata and preset business rules determined after different data sources are filled in a uniform data filling format are obtained; the metadata are used for recording an ontology related to the preset business rule and determining corresponding derivative indexes by utilizing a target ontology in the metadata in an aggregation manner; the target ontology is an ontology selected from the metadata according to user requirements; then, acquiring related data corresponding to the metadata in a preset data acquisition mode, and loading the related data into a corresponding entity; and finally, performing batch rule operation on the entity according to the preset service rule so as to return a corresponding processing result. Therefore, the data access format of the rule engine is unified, relevant data corresponding to the metadata are obtained and loaded as corresponding data entities to be used as data access, and the conventional data input mode facing to the data table is abandoned. The uniformity of data access is guaranteed, different running environments such as online, offline and near-line can be executed by using one set of rule codes, only one set of rule codes needs to be maintained in different environments, different environment data sources can be mixed and used even, and the application range of the rule engine is greatly enlarged. And performing batch rule operation on the entity according to the preset service rule, thereby realizing intensive batch rule calculation and greatly improving the operation efficiency of the service rule. In addition, according to the technical scheme, each data item in each entity is loaded into the memory when the rules are executed, and all derived indexes are processed from the attributes in the ontology, so that which data item does not reach the standard in each business rule can be directly output, all derived indexes can be quickly traced, the data source of the ontology is given, the problem that the rules are separated from the result is solved, and the interpretability is provided for the business application scene.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flow chart of a method for rule engine based data processing as disclosed herein;
FIG. 2 is a schematic data flow diagram of a rules engine based data processing system as disclosed herein;
FIG. 3 is a diagram illustrating exemplary tax domain model data processing according to the present disclosure;
FIG. 4 is a flow chart of a particular rule engine based data processing method disclosed herein;
FIG. 5 is a schematic diagram of a rules engine based data processing system according to the present disclosure;
FIG. 6 is a schematic diagram of a data processing apparatus based on a rules engine according to the present disclosure;
fig. 7 is a block diagram of an electronic device disclosed in the present application.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The existing rule engine has the following defects: (1) lack of interpretability makes the results from this ineffectively applicable; the capability of rule completion is lacked, and special service scenes such as tax benefits and the like cannot be met. (2) The traditional rule engine does not provide a standard for the access data, so that the semantics of the rule engine need to compile rules of different versions in different environments, and the access cost, the maintenance cost and the judgment accuracy cost are high. (3) Distributed operation is limited to rule calculation optimization under the condition of large optimization intervention quantity, and the time consumed after the engine rule is increased is linearly increased, so that the time consumption is long, and the operation efficiency is low.
Therefore, the data processing scheme based on the rule engine is provided, and the interpretability, the data access efficiency and the rule operation efficiency can be improved for a business application scene.
The embodiment of the invention discloses data processing based on a rule engine, and as shown in figure 1, the method comprises the following steps:
step S11: and acquiring metadata and preset business rules determined after different data sources are filled in a uniform data filling format.
Because the traditional rule engines are all data input modes oriented to the data table, no specification is provided for the accessed data, and the semantics of the rule engines need to write different versions of rules in different environments. In the embodiment of the application, the metadata and the preset service rule are determined by facing different data sources and filling the different data sources in a uniform data filling format, so that the data of the metadata definition layer and the preset service rule definition layer are obtained, and further, the related data can be pulled according to the metadata and the preset service rule. The supported data sources comprise a data warehouse, a MYSQL service library, an online interface and the like.
In the embodiment of the application, the metadata is used for recording an ontology related to the preset business rule and determining a corresponding derivative index by aggregating a target ontology in the metadata; the target ontology is an ontology selected from the metadata according to user requirements. For example, if business data related to the tax field needs to be processed currently, the metadata definition layer defines related ontologies and derived indexes according to actual tax rules. If defining that the taxpayer body contains fact elements such as taxpayer type, taxpayer industry, all invoice sets of taxpayers and the like; the invoice body contains the fact elements of taxpayer identification numbers of both sales parties, invoice types, the industry of the sales parties, whether the invoice is a sales invoice and the like.
It is understood that an ontology is a well-known conceptual framework, and in the embodiment of the present application, a policy rule ontology is an attribute included in a policy itself; the invoice body is a related element set with the invoice granularity; the taxpayer body is the related information of the taxpayer such as the file number, the enterprise area information and the like, and the invoice body can be assembled and nested into the taxpayer body.
Further, after the metadata definition of the ontology is completed, metadata of the derived index can be processed and aggregated for the target ontology in the ontology. The derived indexes are data indexes generated by processing and polymerizing the ontology fact elements, such as: indexes such as growth rate and average value are assembled by fact elements, and derived indexes are multiplexed in a plurality of rules, so that all derived indexes can be traced quickly, and the data source of the most source of the ontology is provided. It should be noted that the derived index is derived index processed by aggregation according to the defined attributes in the ontology, and is essentially a calculation logic and rule, and the calculation rule is given when defining the metadata of the derived index, but specific data is not given.
In the embodiment of the application, after the preset service rule is obtained, the preset service rule is disassembled to obtain different rule statements, that is, the preset service rule is disassembled into specific rules according to the actual preset service rule. The rule statements are then configured into corresponding rule grammars for syntax checking and syntax testing of the rule grammars, it being understood that only rules that pass syntax checking and syntax testing can be used and that rules that pass syntax checking and syntax testing are stored for subsequent invocation.
Step S12: and acquiring related data corresponding to the metadata in a preset data acquisition mode, and loading the related data into a corresponding entity.
In the embodiment of the application, after the metadata and the preset business rule are obtained, data filling is carried out according to the ontology in the metadata and the preset business rule. Specifically, the ontology is mapped to fields in different data sources, and different configuration items corresponding to the fields are generated; and performing data packing on the metadata according to the configuration items to obtain an instance corresponding to the ontology. For example, the ontology in the obtained metadata is mapped to fields in an actual database, fields in an interface and the like, and different configuration items are formed to provide the entity access JAVA package, the interface address and the mapping table metadata to the rule engine when the rule engine is accessed in cooperation with different environments, so that the rule engine can obtain ontology instance data in the calculation process.
In the embodiment of the application, the instance is acquired by a preset data acquisition mode, for example, relevant data is acquired by an interface/Map Reduce (programming model) task/JDBC (Java DataBase Connectivity, Java DataBase connection) mode; it can be understood that the acquired related data is data such as a corresponding ontology, ontology instance data filled with data, and a corresponding derivative index, and then, according to configuration items formed by different data sources, corresponding data fetching logic is automatically executed, so that ontologies corresponding to the instances and the instances are loaded as corresponding entities. That is, an entity refers to a combination of an ontology and an instance, such as a policy rule, an invoice data, and a taxpayer data.
In the embodiment of the application, after the related data of the metadata is acquired, the rule engine loads the data. When the rules are loaded, in order to improve the judgment efficiency of the preset business rules on the entities, the loaded target rules are used for carrying out matching judgment on the bodies in different entities one by one so as to carry out batch processing on the entities.
If the matching judgment of the current body and the target rule is completed, a zone bit is placed in the current body, and the target rule is continuously utilized to perform matching judgment with other bodies; and after all the ontology matching judgment is completed, merging the instances which have the same instance and correspond to different target rules, and then pre-compiling the merged target rules in the instances.
Step S13: and performing batch rule operation on the entity according to the preset service rule so as to return a corresponding processing result.
In the embodiment of the application, each loaded entity executes the preset service rule, performs batch rule operation, and then provides data in various forms, such as an interface, a data warehouse, a service database and the like, for the data according to the requirements of a user according to the calculation result.
In the embodiment of the application, the rules are processed in batch when the rules are loaded, so that multiple pieces of rule running test repeated loading data can be changed into single instance unified loading data and batch running test rules when the rules are calculated. Therefore, only one time of loading example data is needed for 1 run of 10 10000 rules, and the original linear time complexity is reduced to a constant level. The performance of one example under the condition of multiple tax policies and multiple tax benefits is greatly improved under the condition that multiple policy checks need to be operated.
Fig. 2 shows the data flow of the whole system when data processing is performed. The rule engine, based on externally input metadata: presetting a service rule, a body and a derivative index required by the rule, pulling related data to load into a corresponding entity in the modes of an interface/Map Reduce task/JDBC and the like according to a configuration item corresponding to an access data source, and performing corresponding rule calculation for each instance. The data sources can be data asset platforms such as an online production service library/an online system interface/a data warehouse, and different data sources are filled in a uniform data filling format, so that the access cost is reduced. And finally, returning the corresponding calculation result to the calling party, the corresponding data warehouse and the corresponding service library by the rule engine so as to support (online, offline and online) running test.
For example, as shown in fig. 3, the taxpayer body, the invoice body and the tax policy body are vertically established in the case of applying the embodiment of the present application to the tax field model. The entity model management platform is responsible for predefining an ontology and complex derivative index calculation logics and rules in the whole link (namely, the derivative indexes processed by aggregation according to the defined attributes in the ontology are essentially calculation logics and rules); tax model management platform: the rule configuration is carried out by utilizing the grammar predefined by the rule engine according to the attribute set of the ontology; a rule engine: providing a distributed rule execution function; providing a data entity loading function; providing a rule completion function; providing a rule interpretive function; data loading layer: according to the ontology defined by the entity model management platform and the standard specification of the rule engine, filling data (providing an interface and a database on line; providing a JAR packet for fetching the entity off line).
According to the method, firstly, metadata and preset business rules determined after different data sources are filled in a uniform data filling format are obtained; the metadata are used for recording an ontology related to the preset business rule and determining a corresponding derivative index by aggregating a target ontology in the metadata; the target ontology is an ontology selected from the metadata according to user requirements; then, acquiring related data corresponding to the metadata in a preset data acquisition mode, and loading the related data into a corresponding entity; and finally, performing batch rule operation on the entity according to the preset service rule so as to return a corresponding processing result. Therefore, the data access format of the rule engine is unified, relevant data corresponding to the metadata are obtained and loaded into corresponding data entities to serve as data access, and the conventional data input mode facing to a data table is abandoned. The uniformity of data access is guaranteed, different running environments such as online, offline and near-line can be executed by using one set of rule codes, only one set of rule codes needs to be maintained in different environments, different environment data sources can be mixed and used even, and the application range of the rule engine is greatly enlarged. And performing batch rule operation on the entity according to the preset service rule, thereby realizing intensive batch rule calculation and greatly improving the operation efficiency of the service rule. In addition, according to the technical scheme of the application, each data item in each entity is loaded into the memory when the rules are executed, and all derived indexes are processed by the attributes in the ontology, so that which data item of each business rule does not reach the standard and can be directly output, all derived indexes can be quickly traced, the data source of the ontology is provided, the separation of the rules and the results is solved, and the interpretability is provided for the business application scene.
The embodiment of the application discloses a specific data processing method based on a rule engine, and as shown in fig. 4, the method includes:
step S21: and obtaining metadata and preset business rules determined after different data sources are filled in a uniform data filling format.
Step S22: and acquiring related data corresponding to the metadata in a preset data acquisition mode, and loading the related data into a corresponding entity.
For more specific processing procedures of the step S21 and the step S22, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
Step S23: and loading the preset business rules, and performing matching judgment on the bodies in different entities one by using the target rules obtained after loading so as to perform batch processing on the entities.
In the embodiment of the application, after the related data of the metadata is acquired, the rule engine loads the data. When the rules are loaded, in order to improve the judgment efficiency of the preset business rules on the entities, the loaded target rules are used for carrying out matching judgment on the bodies in different entities one by one so as to carry out batch processing on the entities. For example, if the taxpayer is Zhang III, Li IV and Wang V, the body conditions to be judged are taxpayer type, the industry to which the taxpayer belongs and tax-containing sales income, if the taxpayer is determined to belong to an individual industrial and commercial business, the industry to which the taxpayer belongs is the service industry and the taxpayer with tax-containing sales income of 2 ten thousand yuan, the body is judged one by one, if the taxpayer type is judged first, the taxpayer meeting the conditions is screened out, then the industry to which the taxpayer belongs is judged, and finally the tax-containing sales income is judged.
Step S24: and if the matching judgment of the current body and the target rule is finished, placing a zone bit on the current body, and continuing to use the target rule to perform matching judgment with other bodies.
It should be noted that, if the matching judgment between the current body and the target rule is completed, a flag bit is placed in the current body, and the target rule is continuously utilized to perform matching judgment with other bodies. For example, if one is three for a taxpayer, there are two pieces of data: one includes three taxpayer types and affiliated industries, the other includes three affiliated industries and sales income with taxes, the data related to the three taxpayer types and affiliated industries are combined to obtain one piece of data including the three taxpayer types, affiliated industries and sales income with taxes, therefore, different target rules of the same example are combined and disambiguated, and the operation efficiency of the rules is improved.
Step S25: and after all the ontology matching judgment is completed, merging the instances which have the same instance and correspond to different target rules, and then pre-compiling the merged target rules in the instances.
In the embodiment of the application, the operation efficiency of the rules can be improved by merging the corresponding target rules with the same examples, and then the operation speed of the rules can be accelerated by pre-compiling the merged rules.
Step S26: and aligning the body appearing in the preset business rule with the body in the entity, and judging whether the entity meets the preset business rule.
In the embodiment of the application, batch rule operation is performed on each loaded entity. Specifically, a preloaded rule, that is, a precompiled target rule, is executed in batch for a loaded entity; then, unlike the past methods, alignment and matching of instances, rules, policies and suggestions based on semantic embedding are performed simultaneously in the rule calculation process, thereby implementing the rule operation of the entity.
Step S27: and if the entity meets the preset service rule, performing batch rule operation on the entity according to the preset service rule to return a corresponding processing result.
When entity judgment is carried out, if the entity meets preset business rules, for example, the taxpayer type with taxpayer three belongs to an individual industrial and commercial customer, the industry belongs to the service industry, and the tax-containing sales income is 2 ten thousand yuan, and when the rules are judged, the taxpayer type belonging to the individual industrial and commercial customer, the industry belongs to the service industry, and the tax-containing sales income is not less than 1 ten thousand yuan is output, the three-part meeting conditions are obtained, corresponding data support is obtained, and data output is carried out on the taxpayer.
Step S28: if the entity does not meet the preset service rule, outputting a data item which does not meet the preset service rule in the entity, and providing a completion suggestion for enabling the entity to meet the preset service rule for the data item.
In the embodiment of the application, after the rule is executed, the reason for meeting or not meeting the condition, including the source policy of the business rule, the preferential rule and the like, is deduced according to the relationship between the input data and the formalized rule. If the entity does not satisfy the preset business rule, for example, in a specific implementation mode, data of Zhang III of taxpayers exist, the data comprises that the taxpayers belong to individual industrial and commercial customers and belong to service industries, tax-containing sales amount of the taxpayers is lacked during rule judgment, the data corresponding to Zhang III belongs to the data lacking the preset business rule judgment condition, and a completion suggestion that the tax-containing sales amount needs to be completed is provided for the data of Zhang III output. In another specific implementation mode, data of taxpayer three-in-one exists, the data comprises that the taxpayer type belongs to an individual industrial business, the industry belongs to a service industry, and the tax-containing sales income is 8000 yuan, if the taxpayer type belongs to the individual industrial business, the industry belongs to the service industry, and the tax-containing sales income is not less than 1 ten thousand yuan, three-in-one does not meet the condition, and the income is less than the income specified by the rule, the suggestion of sales promotion is given to pay the tax.
According to the method, firstly, metadata and preset business rules determined after different data sources are filled in a uniform data filling format are obtained; the metadata are used for recording an ontology related to the preset business rule and determining a corresponding derivative index by aggregating a target ontology in the metadata; the target ontology is an ontology selected from the metadata according to user requirements; then, acquiring related data corresponding to the metadata in a preset data acquisition mode, and loading the related data into a corresponding entity; loading the preset business rules, and performing matching judgment on the bodies in different entities one by using the target rules obtained after loading so as to perform batch processing on the entities; if the matching judgment of the current body and the target rule is finished, placing a zone bit on the current body, and continuing to use the target rule to perform matching judgment with other bodies; after all the ontology matching judgment is completed, merging the instances which have the same instance and correspond to different target rules, and then pre-compiling the merged target rules in the instances; aligning the body appearing in the preset business rule with the body in the entity, and judging whether the entity meets the preset business rule or not; and if the entity meets the preset service rule, performing batch rule operation on the entity according to the preset service rule to return a corresponding processing result. If the entity does not meet the preset service rule, outputting a data item which does not meet the preset service rule in the entity, and providing a completion suggestion for enabling the entity to meet the preset service rule for the data item. Therefore, the data access format of the rule engine is unified, relevant data corresponding to the metadata are obtained and loaded into corresponding data entities to serve as data access, and the conventional data input mode facing to a data table is abandoned. The uniformity of data access is guaranteed, different running environments such as online, offline and near-line can be executed by using one set of rule codes, only one set of rule codes needs to be maintained in different environments, different environment data sources can be mixed and used even, and the application range of the rule engine is greatly enlarged. And performing batch rule operation on the entity according to the preset service rule, thereby realizing intensive batch rule calculation and greatly improving the operation efficiency of the service rule. In addition, according to the technical scheme of the application, each data item in each entity is loaded into the memory when the rules are executed, and all derived indexes are processed by the attributes in the ontology, so that which data item of each business rule does not reach the standard and can be directly output, all derived indexes can be quickly traced, the data source of the ontology is provided, the separation of the rules and the results is solved, and the interpretability is provided for the business application scene.
Fig. 5 is a schematic diagram of a specific system for data processing based on a rule engine. 1. The metadata definition layer is responsible for defining the ontology and the derived indexes. For example, defining the fact elements of taxpayer type, taxpayer industry, all invoice set of taxpayer, etc. in the taxpayer body; defining fact elements such as taxpayer identification numbers of both sellers and sellers, invoice types, seller industries, whether the invoice is a sales invoice and the like in an invoice body; the derived index defines how to process and aggregate the elements of the ontology, gives the calculation rule but does not give specific data, such as X bits before the invoice tax classification code. 2. The rule definition layer is responsible for decomposing into specific rules according to actual tax rules, configuring the rules into grammars predefined by the rule engine, transmitting the rules to the rule engine for grammar check and grammar test, and storing the rules to the rule engine after the rules pass the grammar check and grammar test. 3. The data loading layer is responsible for mapping the ontology in the metadata definition layer into fields in an actual database and fields in an interface, and forming different configuration items so as to provide an entity access JAVA program package, an interface address and mapping table metadata to the rule engine when the rule engine is accessed in cooperation with different environments, so that the rule engine can acquire ontology instance data in the calculation process. 4. And (1) rule loading in the rule engine is responsible for three steps of rule loading, rule condition merging with an instance and rule precompilation. Firstly, expressing and loading a rule in a structured mode into a system; then, carrying out batch processing on the judgment of the example rule, carrying out one-time judgment on the same condition, placing a flag bit, and combining and disambiguating the same example rule to improve the operation efficiency of the rule; and finally, pre-compiling the rule to accelerate the rule running speed. (2) The data loading in the rule engine is used for performing structured representation and reading on data of different data sources. After the module takes the data defined by other layers, different access logics are executed according to different configuration items, and the batch loading of instance data is realized. Meanwhile, when the data is loaded, the module automatically carries out different data access operations on different data sources, and the supported data sources comprise a data warehouse, a MYSQL service library, an online interface and the like. (3) And the rule operation module in the rule engine performs batch rule operation on each loaded entity. Firstly, executing pre-loaded rules in batch aiming at loaded entities; and then, aligning and matching the examples, the rules, the policies and the suggestions based on semantic embedding are carried out simultaneously in the rule calculation process, so that corresponding data support and completion suggestions are obtained based on the examples and the operation rules, data support is output according to the policies, and completion suggestions are given to the policies which are not according to the policies. (4) And the result returning module in the rule engine provides data in various forms (interfaces, data bins and a business database) for the result calculated by the rule engine according to the requirement of a user. (5) The rule engine also comprises other functional item modules which are owned by the traditional rule engine and comprise platform functions of rule grammar check, rule grammar test, rule grammar storage, rule timing tasks, rule execution state monitoring and the like, and the basic service use of the rule engine is met.
It should be noted that, the data loading position may be changed, and the entity definition layer and the data loading layer only provide the definition of the ontology, the processing mode of the derived rules, and the mapping relationship between the ontology elements and the actual data table database interface in order to ensure the decoupling and the running performance between the systems, and are not in charge of data fetching. In addition, the rule engine directly acquires the data table, does not define the entity definition layer and the data loading layer, and essentially acquires the data of the data table, but the rule engine can quickly access different service scenes by using different configurations, so that the service access efficiency is improved, and the rule grammars of the different scenes are unified.
Correspondingly, the embodiment of the present application further discloses a data processing apparatus based on a rule engine, and as shown in fig. 6, the apparatus includes:
the data acquisition module 11 is configured to acquire metadata and preset service rules determined after different data sources are loaded in a uniform data loading format; the metadata are used for recording an ontology related to the preset business rule and determining a corresponding derivative index by aggregating a target ontology in the metadata; the target ontology is an ontology selected from the metadata according to user requirements;
an entity loading module 12, configured to obtain, in a preset data obtaining manner, related data corresponding to the metadata, and load the related data as a corresponding entity;
and the rule operation module 13 is configured to perform batch rule operations on the entity according to the preset service rule so as to return a corresponding processing result.
For more specific working processes of the modules, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
Therefore, according to the scheme of the embodiment, firstly, metadata and preset business rules determined after different data sources are filled in a uniform data filling format are obtained; the metadata are used for recording an ontology related to the preset business rule and determining a corresponding derivative index by aggregating a target ontology in the metadata; the target ontology is an ontology selected from the metadata according to user requirements; then, acquiring related data corresponding to the metadata in a preset data acquisition mode, and loading the related data into a corresponding entity; and finally, performing batch rule operation on the entity according to the preset service rule so as to return a corresponding processing result. Therefore, the data access format of the rule engine is unified, relevant data corresponding to the metadata are obtained and loaded into corresponding data entities to serve as data access, and the conventional data input mode facing to a data table is abandoned. The uniformity of data access is guaranteed, different running environments such as online, offline and near-line can be executed by using one set of rule codes, only one set of rule codes needs to be maintained in different environments, different environment data sources can be mixed and used, and the application range of the rule engine is greatly enlarged. And performing batch rule operation on the entity according to the preset service rule, thereby realizing intensive batch rule calculation and greatly improving the operation efficiency of the service rule. In addition, according to the technical scheme of the application, each data item in each entity is loaded into the memory when the rules are executed, and all derived indexes are processed by the attributes in the ontology, so that which data item of each business rule does not reach the standard and can be directly output, all derived indexes can be quickly traced, the data source of the ontology is provided, the separation of the rules and the results is solved, and the interpretability is provided for the business application scene.
Further, an electronic device is disclosed in the embodiments of the present application, and fig. 7 is a block diagram of an electronic device 20 according to an exemplary embodiment, which should not be construed as limiting the scope of the application.
Fig. 7 is a schematic structural diagram of an electronic device 20 according to an embodiment of the present disclosure. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input output interface 25, and a communication bus 26. Wherein, the memory 22 is used for storing a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the rule engine-based data processing method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in the present embodiment may be a computer.
In this embodiment, the power supply 23 is configured to provide a working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and an external device, and a communication protocol followed by the communication interface is any communication protocol applicable to the technical solution of the present application, and is not specifically limited herein; the input/output interface 25 is configured to obtain external input data or output data to the outside, and a specific interface type thereof may be selected according to specific application requirements, which is not specifically limited herein.
In addition, the memory 22 is used as a carrier for storing resources, and may be a read-only memory, a random access memory, a magnetic disk, an optical disk, or the like, the resources stored thereon may include an operating system 221, a computer program 222, data 223, and the like, and the data 223 may include various data. The storage means may be transient storage or permanent storage.
The operating system 221 is used for managing and controlling each hardware device on the electronic device 20 and the computer program 222, and may be Windows Server, Netware, Unix, Linux, or the like. The computer program 222 may further include a computer program that can be used to perform other specific tasks in addition to the computer program that can be used to perform the rule engine-based data processing method disclosed in any of the foregoing embodiments and executed by the electronic device 20.
Further, embodiments of the present application disclose a computer-readable storage medium, where the computer-readable storage medium includes a Random Access Memory (RAM), a Memory, a Read-Only Memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a magnetic disk, or an optical disk, or any other form of storage medium known in the art. Wherein the computer program, when executed by a processor, implements the aforementioned rule engine-based data processing method. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the foregoing embodiments, which are not described herein again.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
The steps of a rules engine-based data process or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The rule engine-based data processing method, apparatus, device and storage medium provided by the present invention are described in detail above, and specific examples are applied herein to explain the principles and embodiments of the present invention, and the descriptions of the above embodiments are only used to help understand the method and its core ideas of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims (10)

1. A data processing method based on a rule engine is characterized by comprising the following steps:
acquiring metadata and a preset service rule which are determined after different data sources are filled in a uniform data filling format; the metadata are used for recording an ontology related to the preset business rule and determining a corresponding derivative index by aggregating a target ontology in the metadata; the target ontology is an ontology selected from the metadata according to user requirements;
acquiring related data corresponding to the metadata in a preset data acquisition mode, and loading the related data into a corresponding entity;
and performing batch rule operation on the entity according to the preset service rule so as to return a corresponding processing result.
2. The data processing method based on the rule engine as claimed in claim 1, wherein after obtaining the metadata and the preset business rules determined after the different data sources are filled in the uniform data filling format, the method further comprises:
disassembling the preset business rule to obtain different rule sentences;
and configuring the rule statement into a corresponding rule grammar so as to carry out grammar check and grammar test on the rule grammar.
3. The data processing method based on the rule engine as claimed in claim 1, wherein after obtaining the metadata and the preset business rules determined after the different data sources are filled in the uniform data filling format, the method further comprises:
mapping the ontology to fields in different data sources, and generating different configuration items corresponding to the fields;
and performing data packing on the metadata according to the configuration items to obtain an example corresponding to the ontology.
4. The data processing method based on the rule engine as claimed in claim 3, wherein the obtaining of the related data corresponding to the metadata by a preset data obtaining manner and the loading of the related data into the corresponding entity comprises:
and acquiring the instance in a preset data acquisition mode, and automatically executing corresponding access logic according to the configuration item so as to load the instance and the body corresponding to the instance as corresponding entities.
5. The data processing method based on the rule engine as claimed in claim 1, wherein the process of obtaining the related data corresponding to the metadata by a preset data obtaining manner and loading the related data into the corresponding entity further comprises:
and loading the preset business rules, and performing matching judgment on the bodies in different entities one by using the target rules obtained after loading so as to perform batch processing on the entities.
6. The data processing method based on the rule engine as claimed in claim 5, wherein the step of performing matching judgment on the ontologies in different entities one by using the target rule obtained after loading comprises:
if the matching judgment of the current body and the target rule is finished, placing a zone bit on the current body, and continuing to use the target rule to perform matching judgment with other bodies;
and after all the ontology matching judgment is completed, merging the instances which have the same instance and correspond to different target rules, and then pre-compiling the merged target rules in the instances.
7. The data processing method based on the rule engine as claimed in any one of claims 1 to 6, wherein the performing batch rule operation on the entity according to the preset business rule to return a corresponding processing result comprises:
aligning the body appearing in the preset business rule with the body in the entity, and judging whether the entity meets the preset business rule or not;
if the entity meets the preset service rule, performing batch rule operation on the entity according to the preset service rule to return a corresponding processing result;
if the entity does not meet the preset service rule, outputting a data item which does not meet the preset service rule in the entity, and providing a completion suggestion for enabling the entity to meet the preset service rule for the data item.
8. A rules engine based data processing apparatus, comprising:
the data acquisition module is used for acquiring metadata and preset business rules determined after different data sources are filled in a uniform data filling format; the metadata are used for recording an ontology related to the preset business rule and determining a corresponding derivative index by aggregating a target ontology in the metadata; the target ontology is an ontology selected from the metadata according to user requirements;
the entity loading module is used for acquiring related data corresponding to the metadata in a preset data acquisition mode and loading the related data into a corresponding entity;
and the rule operation module is used for performing batch rule operation on the entity according to the preset service rule so as to return a corresponding processing result.
9. An electronic device, comprising a processor and a memory; wherein the memory is for storing a computer program that is loaded and executed by the processor to implement the rules engine based data processing method of any of claims 1 to 7.
10. A computer-readable storage medium for storing a computer program; wherein the computer program when executed by a processor implements a rules engine based data processing method as claimed in any one of claims 1 to 7.
CN202210913579.5A 2022-08-01 2022-08-01 Data processing method, device and equipment based on rule engine and storage medium Active CN114996319B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210913579.5A CN114996319B (en) 2022-08-01 2022-08-01 Data processing method, device and equipment based on rule engine and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210913579.5A CN114996319B (en) 2022-08-01 2022-08-01 Data processing method, device and equipment based on rule engine and storage medium

Publications (2)

Publication Number Publication Date
CN114996319A true CN114996319A (en) 2022-09-02
CN114996319B CN114996319B (en) 2022-11-04

Family

ID=83021512

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210913579.5A Active CN114996319B (en) 2022-08-01 2022-08-01 Data processing method, device and equipment based on rule engine and storage medium

Country Status (1)

Country Link
CN (1) CN114996319B (en)

Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080256047A1 (en) * 2007-04-16 2008-10-16 Dettinger Richard D Selecting rules engines for processing abstract rules based on functionality and cost
AU2011213842A1 (en) * 2010-09-03 2012-03-22 Tata Consultancy Services Limited A system and method of managing mapping information
US20130332404A1 (en) * 2012-06-07 2013-12-12 International Business Machines Corporation Implementing meta rules on an executable rule engine
US20140195473A1 (en) * 2013-01-08 2014-07-10 International Business Machines Corporation Production rule engine
US20140337808A1 (en) * 2013-05-09 2014-11-13 Sap Ag Space Constrained Small Format Visual Analytic Labeling
CN106296399A (en) * 2015-06-11 2017-01-04 交通银行股份有限公司 The data processing method of business rule formulation and system
US20180011912A1 (en) * 2016-07-11 2018-01-11 Al-Elm Information Security Co. Methods and systems for multi-dynamic data retrieval and data disbursement
CN110297840A (en) * 2019-05-22 2019-10-01 平安银行股份有限公司 Data processing method, device, equipment and the storage medium of rule-based engine
CN110888672A (en) * 2019-11-26 2020-03-17 北京仁科互动网络技术有限公司 Metadata architecture-based expression engine implementation method and system
CN111737400A (en) * 2020-06-15 2020-10-02 上海理想信息产业(集团)有限公司 Knowledge reasoning-based big data service tag expansion method and system
CN112948486A (en) * 2021-02-04 2021-06-11 北京淇瑀信息科技有限公司 Batch data synchronization method and system and electronic equipment
CN113360554A (en) * 2020-03-06 2021-09-07 深圳法大大网络科技有限公司 Method and equipment for extracting, converting and loading ETL (extract transform load) data
WO2021188344A1 (en) * 2020-03-18 2021-09-23 Microsoft Technology Licensing, Llc Compliant entity conflation and access
CN113641705A (en) * 2021-08-16 2021-11-12 神州数码融信软件有限公司 Marketing disposal rule engine method based on calculation engine
CN113742493A (en) * 2021-09-01 2021-12-03 志诺维思(北京)基因科技有限公司 Method and device for constructing pathological knowledge map
CN114090653A (en) * 2021-11-17 2022-02-25 中通服软件科技有限公司 Resource data statistical method and device, meta-platform equipment and storage medium
CN114443042A (en) * 2021-12-13 2022-05-06 北京国电通网络技术有限公司 Service arrangement execution method based on rule engine and related equipment
CN114611486A (en) * 2022-03-09 2022-06-10 上海弘玑信息技术有限公司 Information extraction engine generation method and device and electronic equipment

Patent Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080256047A1 (en) * 2007-04-16 2008-10-16 Dettinger Richard D Selecting rules engines for processing abstract rules based on functionality and cost
AU2011213842A1 (en) * 2010-09-03 2012-03-22 Tata Consultancy Services Limited A system and method of managing mapping information
US20130332404A1 (en) * 2012-06-07 2013-12-12 International Business Machines Corporation Implementing meta rules on an executable rule engine
US20140195473A1 (en) * 2013-01-08 2014-07-10 International Business Machines Corporation Production rule engine
US20140337808A1 (en) * 2013-05-09 2014-11-13 Sap Ag Space Constrained Small Format Visual Analytic Labeling
CN106296399A (en) * 2015-06-11 2017-01-04 交通银行股份有限公司 The data processing method of business rule formulation and system
US20180011912A1 (en) * 2016-07-11 2018-01-11 Al-Elm Information Security Co. Methods and systems for multi-dynamic data retrieval and data disbursement
CN110297840A (en) * 2019-05-22 2019-10-01 平安银行股份有限公司 Data processing method, device, equipment and the storage medium of rule-based engine
CN110888672A (en) * 2019-11-26 2020-03-17 北京仁科互动网络技术有限公司 Metadata architecture-based expression engine implementation method and system
CN113360554A (en) * 2020-03-06 2021-09-07 深圳法大大网络科技有限公司 Method and equipment for extracting, converting and loading ETL (extract transform load) data
WO2021188344A1 (en) * 2020-03-18 2021-09-23 Microsoft Technology Licensing, Llc Compliant entity conflation and access
CN111737400A (en) * 2020-06-15 2020-10-02 上海理想信息产业(集团)有限公司 Knowledge reasoning-based big data service tag expansion method and system
CN112948486A (en) * 2021-02-04 2021-06-11 北京淇瑀信息科技有限公司 Batch data synchronization method and system and electronic equipment
CN113641705A (en) * 2021-08-16 2021-11-12 神州数码融信软件有限公司 Marketing disposal rule engine method based on calculation engine
CN113742493A (en) * 2021-09-01 2021-12-03 志诺维思(北京)基因科技有限公司 Method and device for constructing pathological knowledge map
CN114090653A (en) * 2021-11-17 2022-02-25 中通服软件科技有限公司 Resource data statistical method and device, meta-platform equipment and storage medium
CN114443042A (en) * 2021-12-13 2022-05-06 北京国电通网络技术有限公司 Service arrangement execution method based on rule engine and related equipment
CN114611486A (en) * 2022-03-09 2022-06-10 上海弘玑信息技术有限公司 Information extraction engine generation method and device and electronic equipment

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
DI LIU ET AL.: "Rule Engine based on improvement Rete algorithm", 《THE 2010 INTERNATIONAL CONFERENCE ON APPERCEIVING COMPUTING AND INTELLIGENCE ANALYSIS PROCEEDING》 *
汤冬冬: "基于Drools的分布式业务规则引擎的设计与实现", 《中国优秀硕士学位论文全文数据库信息科技辑》 *
高萍等: "规则和流程管理在纳税评估系统中的应用研究", 《计算机工程与设计》 *

Also Published As

Publication number Publication date
CN114996319B (en) 2022-11-04

Similar Documents

Publication Publication Date Title
US7574379B2 (en) Method and system of using artifacts to identify elements of a component business model
CA2508537C (en) Method, system, and apparatus for providing access to workbook models through remote function calls
US9542469B2 (en) Data warehouse data model adapters
US20110145787A1 (en) Business object change management using release status codes
CN102542382A (en) Method and device for managing business rule
US10332010B2 (en) System and method for automatically suggesting rules for data stored in a table
CN116701662A (en) Knowledge graph-based supply chain data management method, device, equipment and medium
US9032362B2 (en) System and method for generating high performance calculators for calculation graphs
CN112604273B (en) Data-driven game system function loading method, device and storage medium
CN112256355A (en) Data-driven system function loading method, equipment and storage medium
CN114996319B (en) Data processing method, device and equipment based on rule engine and storage medium
WO2022253165A1 (en) Scheduling method, system, server and computer readable storage medium
CN113326401B (en) Method and system for generating field blood relationship
US11645283B2 (en) Predictive query processing
CN113138760B (en) Page generation method and device, electronic equipment and medium
US20140149186A1 (en) Method and system of using artifacts to identify elements of a component business model
CN117009397A (en) Data query method, data query device, electronic equipment and storage medium
CN110633459A (en) Method and system for automatically generating data report and computer readable storage medium
CN114328965A (en) Knowledge graph updating method and device and computer equipment
CN113935301B (en) Report generation method and device, storage medium and electronic equipment
CN117648339B (en) Data exploration method and device, server and storage medium
KR101975998B1 (en) Apparatus and Method for Data Migration Based on SQL sentences
CN117234466B (en) Enterprise management software development method, system, equipment and storage medium
CN116955414A (en) Data query method of low-code platform and related system
CN117311724A (en) Tosca analysis realization method, tosca analyzer and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant