CN116501753A

CN116501753A - Data processing method and device based on rule engine and rule engine system

Info

Publication number: CN116501753A
Application number: CN202310466844.4A
Authority: CN
Inventors: 李岩; 苗森
Original assignee: China Everbright Bank Co Ltd
Current assignee: China Everbright Bank Co Ltd
Priority date: 2023-04-26
Filing date: 2023-04-26
Publication date: 2023-07-28

Abstract

The invention provides a data processing method and device based on a rule engine and a rule engine system, wherein the method comprises the following steps: the method comprises the steps of obtaining a rule configuration table and a plurality of service data to be processed, splitting the plurality of judging rules into a plurality of different sub-conditions, carrying out parallel processing on the plurality of service data according to the plurality of sub-conditions to obtain a plurality of sub-judging results of each service data, and respectively determining a plurality of target judging results corresponding to each service data according to the plurality of sub-judging results of each service data. According to the embodiment of the invention, the complex logic judgment condition is split into the condition with the finest granularity, the rule judgment of mass data is realized by means of the natural parallel computing advantage of the big data frame, and the problem that a rule engine in the related technology can support mass data processing less is solved.

Description

Data processing method and device based on rule engine and rule engine system

Technical Field

The present invention relates to the field of data processing, and in particular, to a data processing method and apparatus based on a rule engine, and a rule engine system.

Background

Currently common rule engines mainly break down business decisions from application code and write business decisions using predefined semantic modules. The complete rule engine is mainly composed of three parts: rule bases (Rules), established Facts (Facts), inference engines (information Engine/Rules Engine).

The prior art rules engine also has the following problems: (1) Most of the combination logic can only support simple logic judgment and simple rule priority control, and the configuration of the combination logic is relatively complex; (2) Through a rule engine configured by an interface, the visual operation needs to add rule contents one by one on pages, and the workload is high; (3) The rule engine based on the script language is not easy to locate when the rule is modified, a certain code writing capability is needed, and the use difficulty of business personnel is high; (4) A rule engine developed based on a Java virtual machine (Java Virtual Machine, JVM for short) has certain learning cost for common developers; (5) The prior art is mainly based on a single machine environment, and can support mass data processing less.

At present, no mature rule engine product exists in a big data environment.

Disclosure of Invention

The embodiment of the invention provides a data processing method and device based on a rule engine and a rule engine system, which at least solve the problem that the rule engine in the related art can support mass data processing less.

According to an embodiment of the present invention, there is provided a rule engine-based data processing method including: acquiring a prestored rule configuration table and a plurality of business data to be processed, wherein the rule configuration table comprises a plurality of judging rules; splitting the plurality of decision rules into a plurality of mutually different sub-conditions, wherein each decision rule contains at least one sub-condition; the plurality of business data are processed in parallel according to the plurality of sub-conditions, and a plurality of sub-judgment results of each business data are obtained; and respectively determining a plurality of target judgment results corresponding to each service data according to the plurality of sub judgment results of each service data, wherein each judgment rule corresponds to one target judgment result.

Optionally, determining a plurality of target determination results corresponding to each service data according to the plurality of sub-determination results of each service data, including: for each service data, respectively confirming at least one sub-judging result associated with each judging rule according to the association relation between the judging rules and the sub-conditions, wherein each sub-judging result corresponds to one sub-condition in the judging rules; under the condition that the sub-judging results related to the judging rules are all yes, determining that the target judging result of the judging rules is yes; and if any one sub-judging result associated with the judging rule is NO, determining that the target judging result of the judging rule is NO.

Optionally, before the acquiring the rule configuration table stored in advance and the plurality of service data to be processed, the method further includes: uploading rule files written according to preset specifications to a database, and updating the rule configuration table according to the rule files, wherein the rule files are table files, and the rule files and the rule configuration table contain a plurality of Chinese fields.

Optionally, before the splitting the plurality of decision rules into a plurality of mutually different sub-conditions, the method further comprises: analyzing the rule configuration table, and converting Chinese fields in the rule configuration table into English fields according to a preset word stock, wherein the preset word stock stores the mapping relation between the Chinese fields and the English fields.

Optionally, the rule configuration table contains the following fields: rule number, serial number; each row of data in the rule configuration table corresponds to one sub-condition, each judging rule comprises at least one row of data, the rule number is a unique identifier of the judging rule, and the sequence number is a sequence number of each sub-condition in each judging rule.

Optionally, the splitting the plurality of decision rules into a plurality of mutually different sub-conditions includes: and sequentially numbering each sub-condition in the rule configuration table to obtain a plurality of mutually different sub-conditions, wherein each condition number corresponds to a different sub-condition, and the condition numbers of the same sub-condition are the same.

Optionally, the parallel processing is performed on the plurality of service data according to the plurality of sub-conditions to obtain a plurality of sub-determination results of each service data, including: logic judgment is carried out on the sub-conditions corresponding to each condition number according to the plurality of service data, so that a plurality of sub-judgment results corresponding to each service data are obtained; and storing the plurality of sub-judgment results and the corresponding condition numbers into a database.

Optionally, the logic judgment is performed on the sub-condition corresponding to each condition number according to the plurality of service data to obtain a plurality of sub-judgment results corresponding to each service data, including: the rule configuration table also contains the following fields: according to the fields, rule operators, hit content and discrimination modes; extracting target data from the service data according to the basis field; determining target keywords according to the discrimination modes and the hit content; and determining the sub-judging result according to the target data, the target keyword and the rule operator.

Optionally, the determining the target keyword according to the discrimination mode and the hit content includes: the distinguishing mode comprises the following steps: keywords, word stock codes and other fields in a table, wherein the distinguishing mode is used for indicating the data type of the hit content;

Under the condition that the judging mode is a keyword, directly determining the keyword in the hit content as the target keyword;

searching a related keyword of the word stock code in the hit content from a preset word stock under the condition that the distinguishing mode is the word stock code, and determining the related keyword as the target keyword, wherein the mapping relation between the word stock code and the related keyword is stored in the preset word stock;

and extracting keyword data corresponding to the field in the hit content from the service data under the condition that the judging mode is other fields in the table, and determining the keyword data as the target keyword.

Optionally, the determining the sub-decision result according to the target data, the target keyword and the rule operator includes: determining that the sub-decision result is yes if the relation between the target data and the target keyword meets the rule operator; determining whether the sub-decision result is negative in the case that the relation between the target data and the target keyword does not satisfy the rule operator; the types of the target data and the target keywords comprise character strings and numerical values, and the rule operators comprise character string logic judgment and numerical value logic judgment.

Optionally, before the determining, according to the multiple sub-determination results of each service data, multiple target determination results corresponding to each service data, the method further includes: sorting a plurality of judging rules in the rule configuration table according to the priority from high to low according to the rule priority; according to the rule numbers, sequencing a plurality of judging rules with the same priority according to the sequence from the smaller number to the larger number so as to enable the judging rules to be executed in sequence; wherein the rule configuration table further comprises the following fields: rule priority.

According to another embodiment of the present invention, there is also provided a rule engine-based data processing apparatus including:

the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a prestored rule configuration table and a plurality of business data to be processed, and the rule configuration table comprises a plurality of judging rules;

a splitting module, configured to split the plurality of decision rules into a plurality of mutually different sub-conditions, where each decision rule includes at least one sub-condition;

the processing module is used for carrying out parallel processing on the plurality of business data according to the plurality of sub-conditions to obtain a plurality of sub-judgment results of each business data;

And the determining module is used for respectively determining a plurality of target judging results corresponding to each service data according to the plurality of sub-judging results of each service data, wherein each judging rule corresponds to one target judging result.

According to another embodiment of the present invention, there is also provided a rule engine system including:

a rule base for storing a rule configuration table, wherein the rule configuration table contains a plurality of decision rules, and each decision rule contains at least one sub-condition;

and the inference engine is used for carrying out parallel processing on a plurality of business data according to the steps in any method embodiment.

According to a further embodiment of the present invention, there is also provided a computer-readable storage medium having stored therein a computer program, wherein the computer program, when executed by a processor, performs the steps of any of the method embodiments described above.

According to a further embodiment of the invention, there is also provided an electronic device comprising a memory having stored therein a computer program and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.

The invention can solve the problem that the rule engine in the related technology can support less mass data processing, and realizes the rule judgment of mass data by splitting the complex logic judgment condition into the condition of the finest granularity and by means of the natural parallel computing advantage of the big data frame.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention and do not constitute a limitation on the invention.

FIG. 1 is a block diagram of the hardware architecture of a rule engine based data processing apparatus according to an embodiment of the present invention;

FIG. 2 is a flow chart of a rule engine based data processing method according to an embodiment of the invention;

FIG. 3 is a schematic diagram of a rule configuration table according to an embodiment of the invention;

FIG. 4 is a diagram showing a table structure of a word stock according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a rules engine system according to an embodiment of the invention;

FIG. 6 is a schematic diagram of a logic decision flow of a rules engine in accordance with an embodiment of the present invention;

FIG. 7 is a block diagram of a rule engine based data processing apparatus according to an embodiment of the present invention.

Detailed Description

The invention will be described in detail hereinafter with reference to the drawings in conjunction with embodiments. It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other.

It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order.

Example 1

The method embodiment provided in the first embodiment of the present application may be executed in a mobile terminal, a computer terminal or a similar computing device. Taking a mobile terminal as an example, fig. 1 is a block diagram of a hardware structure of a rule engine-based data processing apparatus according to an embodiment of the present invention, as shown in fig. 1, the mobile terminal may include one or more (only one is shown in fig. 1) processors 12 (the processors 12 may include, but are not limited to, a microprocessor MCU, a programmable logic device FPGA, etc. processing means) and a memory 14 for storing data, and optionally, the mobile terminal may further include a transmission device 16 for communication functions and an input-output device 18. It will be appreciated by those skilled in the art that the structure shown in fig. 1 is merely illustrative and not limiting of the structure of the mobile terminal described above. For example, the mobile terminal may also include more or fewer components than shown in fig. 1, or have a different configuration than shown in fig. 1.

The memory 14 may be used to store computer programs, such as software programs of application software and modules, such as computer programs corresponding to the rule engine-based data processing method in the embodiment of the present invention, and the processor 12 executes the computer programs stored in the memory 14 to perform various functional applications and data processing, i.e., to implement the above-described method. Memory 14 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 14 may further include memory located remotely from processor 12, which may be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission device 16 is used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 16 includes a network adapter (Network Interface Controller, simply referred to as NIC) that can connect to other network devices through a base station to communicate with the internet. In one example, the transmission device 16 may be a Radio Frequency (RF) module for communicating with the internet wirelessly.

In this embodiment, a rule engine-based data processing method running in the mobile terminal or the network architecture is provided, and fig. 2 is a flowchart of the rule engine-based data processing method according to an embodiment of the present invention, as shown in fig. 2, where the flowchart includes the following steps:

step S202, a rule configuration table stored in advance and a plurality of business data to be processed are obtained;

step S204, splitting the plurality of judging rules into a plurality of mutually different sub-conditions;

step S206, processing the plurality of business data in parallel according to the plurality of sub-conditions to obtain a plurality of sub-judgment results of each business data;

step S208, a plurality of target judgment results corresponding to each service data are respectively determined according to the plurality of sub judgment results of each service data.

In this embodiment, the rule configuration table includes a plurality of decision rules, each decision rule including at least one sub-condition, and each decision rule corresponds to a target decision result.

In the embodiment of the invention, the rule judgment of the mass service data can be realized by the methods in the steps S202 to S208, the rule engine is designed based on the big data frame, and the complex service judgment rule is split into the condition of the finest granularity, so that the realization difficulty of service logic is reduced, and the efficiency of parallel processing of the mass data is improved.

Furthermore, when the same sub-condition repeatedly appears in different judging rules, only one sub-condition judgment is needed, so that the logic judging rules are simplified, and the operation amount of a rule engine is reduced.

In this embodiment, step S208 may specifically include:

for each service data, respectively confirming at least one sub-judging result associated with each judging rule according to the association relation between the judging rules and the sub-conditions, wherein each sub-judging result corresponds to one sub-condition in the judging rules;

under the condition that the sub-judging results related to the judging rules are all yes, determining that the target judging result of the judging rules is yes;

and if any one sub-judging result associated with the judging rule is NO, determining that the target judging result of the judging rule is NO.

Further, each judgment rule includes at least one sub-condition, and the sub-conditions under the same judgment rule are in a 'and' relationship, so that each sub-condition needs to be satisfied at the same time to judge the judgment rule as passing, and a corresponding result is returned.

In this embodiment, before step S202, the method further includes: uploading rule files written according to preset specifications to a database, and updating the rule configuration table according to the rule files, wherein the rule files are table files, and the rule files and the rule configuration table contain a plurality of Chinese fields.

Specifically, a business person can write business rule logic in Excel according to a preset specification, the rule file supports Chinese expressions, the business person does not need to write codes, only needs to know the writing specification of the pure text logic, and the written rule file is uploaded to a database for storage after being processed by a specified method, so that iterative updating of the rule can be completed. By the embodiment of the invention, business personnel can quickly iterate the version of the business rule without interfacing with the developer, and the readability of the rule file and the rule configuration table is stronger.

In this embodiment, before step S204, the method further includes: analyzing the rule configuration table, and converting Chinese fields in the rule configuration table into English fields according to a preset word stock, wherein the preset word stock stores the mapping relation between the Chinese fields and the English fields.

Furthermore, when the rule engine analyzes the configuration table, the Chinese fields in the configuration table are firstly converted into English according to Chinese-English mapping in the word stock, so that the normal operation of the code program is ensured.

In one embodiment, the rule configuration table contains the following fields: rule number, serial number; each row of data in the rule configuration table corresponds to one sub-condition, each judging rule comprises at least one row of data, the rule number is a unique identifier of the judging rule, and the sequence number is a sequence number of each sub-condition in each judging rule.

In this embodiment, step S204 may specifically include: and sequentially numbering each sub-condition in the rule configuration table to obtain a plurality of mutually different sub-conditions, wherein each condition number corresponds to a different sub-condition, and the condition numbers of the same sub-condition are the same.

Further, each sub-condition corresponds to a line of data in the rule configuration table, and each sub-condition only contains the simplest logic judgment condition

In this embodiment, step S206 may specifically include:

step S2062, respectively carrying out logic judgment on the sub-conditions corresponding to each condition number according to the plurality of service data to obtain a plurality of sub-judgment results corresponding to each service data;

step S2064, storing the plurality of sub-determination results and the corresponding condition numbers in a database.

Further, in step S208, the target determination result may be determined directly according to the condition number and the corresponding sub-determination result included in each determination rule.

In this embodiment, the rule configuration table also contains the following fields: based on fields, rule operators, hit content, and discriminant.

Further, step S2062 may specifically include: extracting target data from the service data according to the basis field; determining target keywords according to the discrimination modes and the hit content; and determining the sub-judging result according to the target data, the target keyword and the rule operator.

In this embodiment, the discrimination modes may specifically include: keywords, word stock coding, and other fields in the table, wherein the discriminant is used for indicating the data type of the hit content.

Further, determining a target keyword according to the discrimination mode and the hit content includes:

In this embodiment, determining the sub-decision result according to the target data, the target keyword, and the rule operator includes:

Determining that the sub-decision result is yes if the relation between the target data and the target keyword meets the rule operator;

determining whether the sub-decision result is negative in the case that the relation between the target data and the target keyword does not satisfy the rule operator;

the types of the target data and the target keywords comprise character strings and numerical values, and the rule operators comprise character string logic judgment and numerical value logic judgment.

Specifically, the string logical judgment can support logical judgment such as "include, not include, equal to, not equal to, beginning with, ending with" and the like, and all implement logical judgment through regular grammar; the numerical logic judgment can comprise logic judgment such as 'greater than, less than, equal to, not equal to', and the like, and numerical comparison logic such as 'greater than, equal to, less than, absolute value', and the like can be expanded.

Furthermore, the present embodiment is limited by the current service requirement, and only the rule judgment of the data types of the numerical value and the character string can be supported, but the present invention is not limited to the data types of the numerical value and the character string, and the developer can expand the supported data types and the corresponding logic judgment rules according to the service requirement.

In one embodiment, the rule configuration table also contains the following fields: rule priority.

Further, before step S208, the method may further include: sorting a plurality of judging rules in the rule configuration table according to the priority from high to low according to the rule priority; and sequencing the plurality of judging rules with the same priority according to the rule numbers in the order from the smaller number to the larger number so as to enable the judging rules to be executed in order.

Specifically, to solve the problem that the same piece of data satisfies a plurality of rules, a priority policy is adopted to solve the rule conflict. The program is sequenced from small to large according to the rule priority (1-9) and the rule number (left and right numerical values are respectively sequenced after separation by "-", and 0-9 or A-Z) and sequentially executed. And returning the rule number of the corresponding rule after the certain rule is judged to pass.

By the embodiment of the invention, complex business logic can be stripped from the codes, and the realization difficulty of the business logic is reduced; the business personnel only need to know the rule specification, does not need to write codes, and can quickly iterate the version; the complex business logic is split into simple conditions, so that parallel processing of massive business data is supported, and the processing efficiency of the business data is improved.

Example 2

FIG. 3 is a schematic diagram of a rule configuration table according to an embodiment of the present invention, as shown in FIG. 3, wherein the rule configuration table includes a plurality of Chinese fields, and the rule is determined to be a Chinese expression.

In this embodiment, the rule configuration table may specifically include the following fields: rule number, sequence number, rule operator, discriminant, hit content, rule priority.

In this embodiment, a complete rule may involve multiple sub-conditions, each rule is uniquely identified by a "rule number," which is the sequential number of different sub-conditions associated with the same rule, and each row of data in the rule configuration table corresponds to one sub-condition.

Further, each rule number is a complete rule condition, rules with different serial numbers under the same rule number are in 'and' relationship, and rule conditions of all serial numbers are met, namely, rule judgment of the rule number is considered to pass, and corresponding results are returned.

In this embodiment, the original field in the data table (service data) [ according to field ]; the "rule operator" is an operation mode of rule judgment, and supports various judgment modes; [ Distinguishing ] means determining the source of content and is also used to indicate the type of data hitting the content; the hit content has a plurality of formats such as numerical values, keywords, word stock codes and the like, and the program processes the information in the hit content according to the discrimination mode.

Specifically, the "discriminant" may be "word stock coding", "keyword" or "numerical value". When the discrimination mode is word stock coding, the program associates corresponding keywords from the word stock according to the word stock coding (such as 0110800) in the word stock (hit content); when the judgment mode is "keyword" or "numerical value", the program directly reads the data in the hit content.

Further, each hit content may include a plurality of information, and the information is in a "divided" or "relationship, that is, when the content to be judged satisfies any one of the conditions [ hit content ], the rule is judged to pass.

In this embodiment, to solve the condition that the same piece of data satisfies a plurality of rules, a priority policy is adopted to solve the rule conflict. The program is sequenced from small to large according to the (rule priority) of (from 1 to 9) and (rule number) of (left side numerical value and right side numerical value are respectively sequenced after separation by "-", and 0 to 9, A to Z) and sequentially executed. And returning the rule number of the corresponding rule after the certain rule is judged to pass.

In this embodiment, the service personnel only needs to fill the rule content into the fields of [ rule number ], [ sequence number ], [ according to the field ], [ rule operator ], [ discriminant manner ], [ hit content ], [ rule priority ], etc. according to the service specification in Excel. The sequence numbers of the rules are not different in sequence, and the sub-rules are added only by inserting one row after the original rules, and the sequence numbers are increased by one. When the problem of error of rule content occurs, the original rule can be directly modified.

Further, according to the service design requirement, the deletion rule can directly delete the whole row or use the [ rule valid mark ] and the [ rule modification date ] fields for version control.

In this embodiment, after the modification of the rule file is completed, the validity of the rule needs to be verified by using a verification program, so that the logic error in the rule can be directly displayed to the service personnel to facilitate the modification of the error while avoiding the error reporting of the program execution.

In this embodiment, the fields are columns in the database, and each record is composed of a plurality of columns for describing a certain characteristic of the data (e.g., opposite party name, abstract, account name, etc.).

Further, the name of the opposite party is the name of the transaction object, and may be the name of the person, the name of the enterprise, or the name meeting a certain specification; the abstract is a simple summary of the data content, which can be formatted data generated by a system or information actively filled by a client; the account name is the name of the customer, and can be a person name, an enterprise name and a name meeting a certain specification.

The embodiment of the invention is not limited to the fields, and the field content and rule writing specification can be designed according to actual service requirements.

Fig. 4 is a table structure diagram of a word stock in the embodiment of the present invention, as shown in fig. 4, the word stock stores mapping relations between chinese fields and english fields, and mapping relations between code values of word stock codes and keywords.

In this embodiment, since there may be a large amount of text information required for determining the actual service requirement, in order to simplify the configuration difficulty of the rule and improve the reusability of the part having the same attribute information, the reusable part is stored in the word library, and is represented by the code value of the word library code when the rule is configured, and the program performs information analysis according to the corresponding code value.

Furthermore, since the rule supports writing in chinese, some key parts of the program do not support chinese information when executing, so that the corresponding relation between the chinese and english names of the key parts needs to be stored in the word stock.

In this embodiment, the specific content of each field in the word stock needs to be added according to the actual service. Fig. 4 is merely an example of a table structure, and the specific data content is not limited.

Example 3

According to another embodiment of the present invention, there is also provided a rule engine system.

FIG. 5 is a schematic diagram of a rule engine system according to an embodiment of the present invention, and as shown in FIG. 5, the complete rule engine system is mainly composed of three parts: rule base 52, fact store 54, inference engine 56.

Rule base 52 (Rules repository), production memory (Production Memory), contains all Rules (Rules) that need to be implemented in the rule engine system.

The fact storage 54, i.e., working Memory (Working Memory), is used to record facts (facts) obtained from an external system or application.

The Inference Engine 56, the Inference Engine (information Engine), uses a Pattern Matcher (Pattern match) to match rules in the rule base with facts in the fact store to determine which rule or rules were triggered.

Further, when there is a conflict between rules to be executed, the conflict needs to be resolved, and the rules are put into Agenda (Agenda) sequentially, for example, in a manner of setting priority of the rules, or sequentially executed according to the rule number. The program will execute all rules in the agenda in turn.

In this embodiment, the rule base 52 is configured to store a rule configuration table, where the rule configuration table includes a plurality of decision rules, and each decision rule includes at least one sub-condition;

in this embodiment, the inference engine 56 is configured to process a plurality of facts (i.e. service data) in parallel according to the method in any of the method embodiments described above.

In the present embodiment, the fact memory 54 is used for storing service data to be processed.

In one embodiment of the present invention, the rule engine system relies on Hadoop clusters, spark2.3 (and above) frameworks and Python3.6 (and above) environments, uses Python (an object-oriented computer programming language) to connect the database to load information of word and rule libraries, analyzes the classified rule logic through Python program processing, and processes the data under pypad frameworks (Python's API under Spark frameworks) to obtain a final classification result.

Fig. 6 is a schematic diagram of a logic determination flow of a rule engine according to an embodiment of the present invention, as shown in fig. 6, the rule engine performs logic determination on all the sub-conditions after splitting, performs secondary determination according to the sub-condition determination result according to the rule execution sequence after parsing, and returns a final determination result of each rule.

The bottom logic in the embodiment is implemented by using hive SQL grammar, and compared with the traditional business rule code development mode, the method has better stability, higher efficiency and lower resource cost in a big data environment.

Furthermore, because the big data frame has the problem of difficult analysis on the calculation of complex logic, the embodiment of the invention can disassemble complex logic judgment conditions to reduce the analysis complexity of the program, and the initial operation is carried out on all the conditions with the finest granularity by the aid of the natural parallel calculation advantages of the big data frame, and then the simple condition judgment is carried out according to the original rule logic execution sequence to obtain a final result.

In this embodiment, the program numbers each row of sub-rules (sub-conditions) one by one (rule_num), the repeated rule numbers are the same, and the judgment results (1, 0, regular matching results) of all the sub-rule conditions are stored in the database. And judging the final result according to the priority order again by the result of the last step to obtain the final result.

In this embodiment, the logic determination flow in fig. 6 splits three complete rules into five mutually different sub-conditions, determines the sub-determination result of each sub-condition (condition a to condition E) respectively, and combines the sub-determination results according to the original rule determination logic to obtain the final determination result. For example, the rule one includes a condition a, a condition B, and a condition D, and when three conditions are simultaneously determined to pass, a determination result of the rule one, that is, a result 1 can be obtained.

In this embodiment, commonly used field types include a string type and a numeric type, and date and time stamps can be converted into a string type by ETL (Extract-Transform-Load).

Specifically, for the numeric field, logic judgment of greater than, less than, equal to, and not equal to is supported (numeric comparison logic of greater than or equal to, less than or equal to, absolute value, etc. can be expanded); for the string type field, logic judgment including, not including, equal to, not equal to, beginning with, ending with, and the like is supported, and all logic judgment is realized through a regular grammar.

In this embodiment, when the rule is configured, fields such as "rule number", "serial number", "basis field", "rule operator", "discriminant", "hit content", "rule priority", etc. are filled in.

In this embodiment, for the sub-condition that the "discrimination method" is the "keyword", the information of the "hit content" is divided into list types by "and each element in the list is equal to or different from the list type (each element is added with" +"" at the beginning), and adding "$") at the end, including or not (without operation), beginning (each element adds "≡at the beginning), ending (each element adds" $ ") at the end, and finally splicing the processed list again by using" | "as a separator to form the regular grammar package of the character string content entering the subsequent flow.

In this embodiment, for the sub-condition that the "word stock coding" is the "discriminant", the code value of the "hit content" is first added to the list one by one from the "word stock" according to all keywords (key_word) of the corresponding code value (lexicon_code), and the following processing logic is the "keyword" with the "discriminant".

In this embodiment, for the sub-condition "other fields in the table" of [ discriminant ], the sub-condition is used to perform logical judgment between two fields, the numeric field supports judgment logic of greater than, less than, equal to, and not equal to, and the string type field supports equal to, not equal to, contains, and does not contain.

In this embodiment, since the field names in the database are all english characters, when the rule engine parses the configuration table, the rule engine needs to convert the chinese-english mapping in the configuration table according to the "word stock" into english, and the "hit content" in which the "discriminant" is "other fields in the table" also needs to be converted into english.

The rule engine in the embodiment of the invention has the following advantages: (1) Complex rule determination combined by superposition of a plurality of simple rules and multi-level rule priority design can be supported; (2) The business rule change is realized through the adjustment of the literal logic of the configuration table, so that the business personnel can maintain the business rule easily, the code writing is not needed, and the code part change is not needed; (3) The logic of the code part is simpler, easy to maintain and easy to expand; (4) Based on Hadoop clusters, the operation processing of the sea data volume is supported, and the authorized use is convenient; and (5) the Chinese expression can be supported, and the readability is high.

In another embodiment, the data flow of the rules engine may use a common platform without developing separate front-end interaction functions. For example, the embodiment can use a big data interaction platform Hue platform to manage a rule base, and use a task scheduling platform Azkaban platform to perform code online and scheduling. The permission management can be realized by limiting the use permission of the user table by an administrator.

In the embodiment of the invention, as the dispatching tool is needed to be used for the online of the cluster environment codes, the code logic is needed to be re-used for the code configuration after the adjustment, and when a complete business process platform is absent, business personnel usually lack the corresponding dispatching tool use authority. When the logic adjustment and iteration are met, cross-department cooperation is needed, the flow is complicated, and multi-stage approval is needed. The embodiment of the invention only needs light simple judgment logic, and can directly write the judgment logic in the configuration file by using hive grammar. When the program is executed, the analysis of the service logic is realized by loading the configuration table, and compared with the mode of directly modifying the code to adjust the service logic, the rule engine system in the embodiment of the invention has lower maintenance cost and higher efficiency.

According to another embodiment of the present invention, there is also provided a rule engine-based data processing apparatus.

FIG. 7 is a block diagram of a rule engine based data processing apparatus according to an embodiment of the present invention, as shown in FIG. 7, comprising:

an obtaining module 72, configured to obtain a rule configuration table and a plurality of service data to be processed, where the rule configuration table includes a plurality of decision rules;

A splitting module 74, configured to split the plurality of decision rules into a plurality of mutually different sub-conditions, where each decision rule includes at least one sub-condition;

a processing module 76, configured to perform parallel processing on the plurality of service data according to the plurality of sub-conditions, so as to obtain a plurality of sub-determination results of each service data;

and a determining module 78, configured to determine a plurality of target determination results corresponding to each service data according to the plurality of sub-determination results of each service data, where each determination rule corresponds to one target determination result.

Optionally, the determining module 78 is further configured to, for each service data, respectively determine at least one sub-determination result associated with each determination rule according to an association relationship between the plurality of determination rules and the plurality of sub-conditions, where each sub-determination result corresponds to one sub-condition in the determination rule; under the condition that the sub-judging results related to the judging rules are all yes, determining that the target judging result of the judging rules is yes; and if any one sub-judging result associated with the judging rule is NO, determining that the target judging result of the judging rule is NO.

Optionally, the apparatus further comprises a configuration module.

The configuration module is used for uploading rule files written according to preset specifications to a database, and updating the rule configuration table according to the rule files, wherein the rule files are table files, and the rule files and the rule configuration table contain a plurality of Chinese fields.

Optionally, the apparatus further comprises a parsing module.

The analysis module is used for analyzing the rule configuration table and converting Chinese fields in the rule configuration table into English fields according to a preset word stock, wherein the preset word stock stores the mapping relation between the Chinese fields and the English fields.

Optionally, the splitting module 74 is further configured to sequentially perform condition numbering on each sub-condition in the rule configuration table, so as to obtain the plurality of mutually different sub-conditions, where each condition number corresponds to a different sub-condition, and the condition numbers of the same sub-condition are the same.

Optionally, the processing module 76 is further configured to perform logic judgment on the sub-condition corresponding to each condition number according to the plurality of service data, so as to obtain a plurality of sub-judgment results corresponding to each service data; and storing the plurality of sub-judgment results and the corresponding condition numbers into a database.

Optionally, the processing module 76 may specifically include:

the extraction unit is used for extracting target data from the business data according to the basis field;

the determining unit is used for determining target keywords according to the judging mode and hit content;

the judging unit is used for determining a sub-judging result according to the target data, the target keywords and the rule operators, wherein the rule configuration table further comprises the following fields: based on fields, rule operators, hit content, and discriminant.

Optionally, the determining unit is further configured to directly determine, when the discrimination manner is a keyword, the keyword in the hit content as the target keyword; searching a related keyword of the word stock code in the hit content from a preset word stock under the condition that the distinguishing mode is the word stock code, and determining the related keyword as the target keyword, wherein the mapping relation between the word stock code and the related keyword is stored in the preset word stock; extracting keyword data corresponding to fields in the hit content from the service data under the condition that the judging mode is other fields in a table, and determining the keyword data as the target keyword; the distinguishing mode comprises the following steps: keywords, word stock coding, and other fields in the table, wherein the discriminant is used for indicating the data type of the hit content.

Optionally, the determining unit is further configured to determine that the sub-determination result is yes, in a case where a relationship between the target data and the target keyword satisfies the rule operator; determining whether the sub-decision result is negative in the case that the relation between the target data and the target keyword does not satisfy the rule operator; the types of the target data and the target keywords comprise character strings and numerical values, and the rule operators comprise character string logic judgment and numerical value logic judgment.

Optionally, the apparatus further comprises a conflict resolution module.

The conflict resolution module is used for sequencing a plurality of judging rules in the rule configuration table according to the priority of the rules from high to low; according to the rule numbers, sequencing a plurality of judging rules with the same priority according to the sequence from the smaller number to the larger number so as to enable the judging rules to be executed in sequence; wherein the rule configuration table further comprises the following fields: rule priority.

It should be noted that each of the above modules may be implemented by software or hardware, and for the latter, it may be implemented by, but not limited to: the modules are all located in the same processor; alternatively, the above modules may be located in different processors in any combination.

Embodiments of the present invention also provide a computer-readable storage medium having a computer program stored therein, wherein the computer program, when executed by a processor, performs the steps of any of the method embodiments described above.

Alternatively, in the present embodiment, the above-described storage medium may be configured to store a computer program for performing the steps of:

s1, acquiring a prestored rule configuration table and a plurality of business data to be processed, wherein the rule configuration table comprises a plurality of judging rules;

s2, splitting the plurality of judging rules into a plurality of mutually different sub-conditions, wherein each judging rule comprises at least one sub-condition;

s3, carrying out parallel processing on the plurality of service data according to the plurality of sub-conditions to obtain a plurality of sub-judgment results of each service data;

s4, respectively determining a plurality of target judgment results corresponding to each service data according to the plurality of sub judgment results of each service data, wherein each judgment rule corresponds to one target judgment result.

Alternatively, in the present embodiment, the storage medium may include, but is not limited to: a usb disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing a computer program.

An embodiment of the invention also provides an electronic device comprising a memory having stored therein a computer program and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.

Optionally, the electronic apparatus may further include a transmission device and an input/output device, where the transmission device is connected to the processor, and the input/output device is connected to the processor.

Alternatively, in the present embodiment, the above-described processor may be configured to execute the following steps by a computer program:

Alternatively, specific examples in this embodiment may refer to examples described in the foregoing embodiments and optional implementations, and this embodiment is not described herein.

It will be appreciated by those skilled in the art that the modules or steps of the invention described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may alternatively be implemented in program code executable by computing devices, so that they may be stored in a memory device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than that shown or described, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps within them may be fabricated into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method of rule engine based data processing comprising:

acquiring a prestored rule configuration table and a plurality of business data to be processed, wherein the rule configuration table comprises a plurality of judging rules;

splitting the plurality of decision rules into a plurality of mutually different sub-conditions, wherein each decision rule contains at least one sub-condition;

the plurality of business data are processed in parallel according to the plurality of sub-conditions, and a plurality of sub-judgment results of each business data are obtained;

and respectively determining a plurality of target judgment results corresponding to each service data according to the plurality of sub judgment results of each service data, wherein each judgment rule corresponds to one target judgment result.

2. The method according to claim 1, wherein the determining a plurality of target decision results corresponding to each service data according to the plurality of sub decision results of each service data, respectively, comprises:

3. The method of claim 1, wherein prior to the obtaining the pre-stored rule configuration table and the plurality of service data to be processed, the method further comprises:

uploading rule files written according to preset specifications to a database, and updating the rule configuration table according to the rule files, wherein the rule files are table files, and the rule files and the rule configuration table contain a plurality of Chinese fields.

4. A method according to claim 3, wherein prior to said splitting the plurality of decision rules into a plurality of mutually different sub-conditions, the method further comprises:

analyzing the rule configuration table, and converting Chinese fields in the rule configuration table into English fields according to a preset word stock, wherein the preset word stock stores the mapping relation between the Chinese fields and the English fields.

5. The method of claim 3, wherein the step of,

the rule configuration table contains the following fields: rule number, serial number;

each row of data in the rule configuration table corresponds to one sub-condition, each judging rule comprises at least one row of data, the rule number is a unique identifier of the judging rule, and the sequence number is a sequence number of each sub-condition in each judging rule.

6. The method of claim 5, wherein splitting the plurality of decision rules into a plurality of mutually different sub-conditions comprises:

and sequentially numbering each sub-condition in the rule configuration table to obtain a plurality of mutually different sub-conditions, wherein each condition number corresponds to a different sub-condition, and the condition numbers of the same sub-condition are the same.

7. The method of claim 6, wherein the parallel processing the plurality of service data according to the plurality of sub-conditions to obtain a plurality of sub-decision results of each service data comprises:

logic judgment is carried out on the sub-conditions corresponding to each condition number according to the plurality of service data, so that a plurality of sub-judgment results corresponding to each service data are obtained;

And storing the plurality of sub-judgment results and the corresponding condition numbers into a database.

8. The method of claim 7, wherein the performing logic judgment on the sub-condition corresponding to each condition number according to the plurality of service data to obtain a plurality of sub-judgment results corresponding to each service data comprises:

the rule configuration table also contains the following fields: according to the fields, rule operators, hit content and discrimination modes;

extracting target data from the service data according to the basis field;

determining target keywords according to the discrimination modes and the hit content;

and determining the sub-judging result according to the target data, the target keyword and the rule operator.

9. The method of claim 8, wherein said determining a target keyword based on said discriminant and said hit content comprises:

the distinguishing mode comprises the following steps: keywords, word stock codes and other fields in a table, wherein the distinguishing mode is used for indicating the data type of the hit content;

10. The method of claim 8, wherein the determining the sub-decision result from the target data, the target keyword, and the rule operator comprises:

11. The method of claim 5, wherein prior to said determining a plurality of target decision results for each service data based on said plurality of sub-decision results for each service data, respectively, the method further comprises:

the rule configuration table also contains the following fields: rule priority;

sorting a plurality of judging rules in the rule configuration table according to the priority from high to low according to the rule priority;

and sequencing the plurality of judging rules with the same priority according to the rule numbers in the order from the smaller number to the larger number so as to enable the judging rules to be executed in order.

12. A rules engine-based data processing apparatus, comprising:

13. A rule engine system, comprising:

an inference engine for parallel processing of a plurality of traffic data according to the method of any of the claims 1 to 11.

14. A computer-readable storage medium, characterized in that a computer program is stored in the storage medium, wherein the computer program, when being executed by a processor, performs the method of any one of claims 1 to 11.

15. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to run the computer program to perform the method of any of the claims 1 to 11.