CN110502483B - Data processing method, data processing device, computer equipment and storage medium - Google Patents

Data processing method, data processing device, computer equipment and storage medium Download PDF

Info

Publication number
CN110502483B
CN110502483B CN201910614641.9A CN201910614641A CN110502483B CN 110502483 B CN110502483 B CN 110502483B CN 201910614641 A CN201910614641 A CN 201910614641A CN 110502483 B CN110502483 B CN 110502483B
Authority
CN
China
Prior art keywords
data
file
engine
processed
component
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910614641.9A
Other languages
Chinese (zh)
Other versions
CN110502483A (en
Inventor
朱传曦
邬稳
韩建
韦芊成
霍亚勇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhaolian Consumer Finance Co ltd
Original Assignee
Merchants Union Consumer Finance Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Merchants Union Consumer Finance Co Ltd filed Critical Merchants Union Consumer Finance Co Ltd
Priority to CN201910614641.9A priority Critical patent/CN110502483B/en
Publication of CN110502483A publication Critical patent/CN110502483A/en
Application granted granted Critical
Publication of CN110502483B publication Critical patent/CN110502483B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/11File system administration, e.g. details of archiving or snapshots
    • G06F16/116Details of conversion of file system types or formats
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/16File or folder operations, e.g. details of user interfaces specifically adapted to file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00Finance; Insurance; Tax strategies; Processing of corporate or income taxes

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Economics (AREA)
  • Accounting & Taxation (AREA)
  • Development Economics (AREA)
  • Human Computer Interaction (AREA)
  • Finance (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • Technology Law (AREA)
  • General Business, Economics & Management (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application relates to a data processing method, a data processing device, computer equipment and a storage medium. The method comprises the following steps: acquiring characteristic information of data to be processed, and acquiring a strategy rule file in a first engine corresponding to the data to be processed according to the characteristic information; converting the policy rule file in the first engine into a script file in a second engine; the first engine is used for writing a strategy rule file, and the second engine is used for executing the strategy rule file in the first engine; and the second engine processes the data to be processed according to the script file, so that the data processing and calculating performance is improved.

Description

Data processing method, data processing device, computer equipment and storage medium
Technical Field
The present application relates to the field of internet technologies, and in particular, to a data processing method and apparatus, a computer device, and a storage medium.
Background
With the rapid development and widespread application of computer and information technologies, data generated by applications in various industries is explosively increased, and data processing technologies are developed in order to ensure the normal operation of applications in various industries. The data processing technology can analyze and process a large amount of data to obtain the data value.
At present, more services are expanded compared with traditional finance by internet finance, real-time performance of data processing is studied by the internet finance, a large number of services and complex processes of the internet finance are solved, and the computing performance of a decision engine for data processing cannot meet the requirement of real-time performance.
Disclosure of Invention
In view of the above, it is necessary to provide a data method, an apparatus, a computer device and a storage medium capable of improving the computing performance.
A method of data processing, the method comprising:
acquiring data to be processed;
acquiring characteristic information of the data to be processed, and acquiring a policy rule file in a first engine corresponding to the data to be processed according to the characteristic information;
converting the policy rule file in the first engine into a script file in a second engine; the first engine is used for writing a policy rule file; the second engine is used for executing the policy rule file in the first engine;
and processing the data to be processed according to the script file through the second engine.
In one embodiment, the acquiring the data to be processed includes:
acquiring the data to be processed through an input interface;
the data to be processed comprises application environment data or comprises application environment data and user data.
In one embodiment, the obtaining the feature information of the data to be processed and obtaining the policy rule file in the first engine corresponding to the data to be processed according to the feature information includes:
acquiring application environment data in the data to be processed;
and acquiring the environment type of the application environment data, and acquiring a policy rule file in the first engine corresponding to the data to be processed according to the environment type.
In one embodiment, the converting the policy rules file in the first engine to the script file in the second engine includes:
acquiring the version parameter of the first engine and the policy rule file identifier selected in a policy rule conversion interface;
acquiring a corresponding policy rule file according to the policy rule file identifier;
acquiring a corresponding file conversion rule according to the version parameter;
and converting the strategy rule file by adopting the file conversion rule to obtain a script file in a second engine.
In one embodiment, the policy rule file is converted by using the file conversion rule to obtain a script file in a second engine;
acquiring a file node structure of the policy rule file;
and converting the strategy rule file in the first engine according to the file node structure to obtain a script file in a second engine.
In one embodiment, the processing the data to be processed according to the script file includes:
acquiring a component list in the script file, and acquiring components in the component list;
calling a component script file corresponding to the component function processing to obtain a script execution file;
and processing the data to be processed according to the script execution file.
In one embodiment, the calling a component script file corresponding to the component function processing corresponding to the component to obtain a script execution file includes:
calling a component script file corresponding to the component function processing;
when the current component function finishes processing the corresponding component script file, acquiring the output arrow direction corresponding to the component;
when a corresponding arrow tail direction value is obtained according to the output arrow direction, calling a next component function in the current component function;
when the corresponding arrow tail direction value is not obtained according to the output arrow direction, jumping out of the current component function, and executing the next component function;
and when the component script files corresponding to the components in the component list are processed, obtaining script execution files.
A data processing apparatus, the apparatus comprising:
the data acquisition module is used for acquiring data to be processed;
the policy rule file acquisition module is used for acquiring a policy rule file in a first engine corresponding to the data to be processed according to the characteristic information of the data to be processed;
the file processing module is used for converting the strategy rule file in the first engine into a script file in a second engine;
and the data processing module is used for processing the data to be processed according to the script file in the second engine.
In one embodiment, the apparatus further comprises:
and the parameter determining module is used for determining the version parameter of the first engine and the policy rule file identifier.
In one embodiment, the apparatus further comprises:
and the extraction module is used for extracting the file node structure of the strategy rule file.
In one embodiment, the apparatus further comprises:
the component acquisition module is used for acquiring components in a component list in the script file;
and the function calling module is used for calling the component function corresponding to the component.
In one embodiment, the apparatus further comprises:
the judging module is used for judging that when the output arrow direction obtains the corresponding arrow tail direction value, the next component function is called in the current component function;
and when the output arrow direction does not acquire the corresponding arrow tail direction value, jumping out of the current component function and executing the next component function.
A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
acquiring data to be processed;
acquiring characteristic information of the data to be processed, and acquiring a policy rule file in a first engine corresponding to the data to be processed according to the characteristic information;
converting the policy rule file in the first engine into a script file in a second engine; the first engine is used for writing a policy rule file, and the second engine is used for executing the policy rule file in the first engine;
and processing the data to be processed according to the script file.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
acquiring data to be processed;
acquiring characteristic information of the data to be processed, and acquiring a policy rule file in a first engine corresponding to the data to be processed according to the characteristic information;
converting the policy rule file in the first engine into a script file in a second engine; the first engine is used for writing a policy rule file, and the second engine is used for executing the policy rule file in the first engine;
and processing the data to be processed according to the script file.
According to the data processing method, the data processing device, the computer equipment and the storage medium, the characteristic information of the data to be processed is obtained, the strategy rule file in the first engine corresponding to the data to be processed is obtained according to the characteristic information, and the corresponding strategy rule file is obtained from the strategy file library in the first engine according to the characteristic information of the data, so that the data obtaining time can be shortened; and converting the strategy rule file in the first engine into a script file in a second engine, wherein the first engine is used for writing the strategy rule file, the second engine is used for processing the data to be processed according to the script file, and the second engine is used for processing the data to be processed independently, so that the calculation performance of the data processing process is improved.
Drawings
FIG. 1 is a diagram of an application environment of a data processing method in one embodiment;
FIG. 2 is a flow diagram illustrating a data processing method according to one embodiment;
FIG. 3 is a flow chart showing the overall design of data processing in another embodiment;
FIG. 4 is a diagram of a policy rules file structure in one embodiment;
FIG. 5 is a diagram of a policy rules file transformation interface in the first engine in one embodiment;
FIG. 6 is a block diagram showing the structure of a data processing apparatus according to an embodiment;
FIG. 7 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The data processing method provided by the application can be applied to the application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The terminal 102 receives data to be processed sent by a server, acquires characteristic information of the data to be processed, and acquires a policy rule file in a first engine corresponding to the data to be processed according to the characteristic information; converting the policy rule file in the first engine into a script file in a second engine; the first engine is used for writing a policy rule file, and the second engine is used for converting the policy rule file in the first engine; and processing the data to be processed according to the script file. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented by an independent server or a server cluster formed by a plurality of servers.
In an embodiment, as shown in fig. 2, a data processing method is provided, which is described by taking the application of the method to the terminal in fig. 1 as an example, and includes the following steps:
step 202, data to be processed is obtained.
The data to be processed is data to be processed. The data to be processed may be financial data, network credit data, company yearbook, social platform data, new media data, etc. For example, the data to be processed is network credit data, wherein the network credit data may include registration data, login data, debit data, transaction data, and the like. The data to be processed may be social platform data, and the social platform data may be data of platform interaction. The data form of the data to be processed may be numerical values, characters, images, sounds, and the like. The data source of the data to be processed can be locally stored data or data acquired from the outside. The externally acquired data can be data downloaded from a network, or data downloaded from a server, or data transmitted by other terminals. The data to be processed is taken as transaction data for example, during transaction, the transaction amount, the information of the transaction user, the selected transaction platform, and the like are involved, the data to be processed may be amount data, user information data, and transaction platform data, and the manner of acquiring the data to be processed may be acquired through an input interface of a decision engine, where the acquisition manner is not limited.
And 204, acquiring the characteristic information of the data to be processed, and acquiring a policy rule file in the first engine corresponding to the data to be processed according to the characteristic information.
The feature information is information having a marking function, and is capable of distinguishing different data. The characteristic information may be an application environment type, which may include a web page, an application program, an applet, a light application, etc., without being limited thereto. For example, taking a financial transaction service as an example, financial transaction service data is acquired, and an application environment type of the financial transaction service is determined. The financial transaction service can complete the financial transaction service in different application environments such as a webpage, a WeChat applet, a Paibao application service and the like through transaction APP and website access. The application environment types of the financial transaction business comprise transaction APP, website access on a webpage, WeChat small programs and Paibao application service.
Taking video data as an example, the to-be-processed data of the obtained video access amount may include application environment data, the application environment of the video access may be characteristic information of the to-be-processed data of the video access amount, the video access may be through vacation APP, microblog, WeChat, and the like, and the application environment types of the to-be-processed data of the video access amount include vacation APP, microblog, WeChat. And acquiring a corresponding policy file in the first engine according to different application environments.
The first engine is a decision engine with high expansibility and strong flexibility, and supports input and output interfaces of different data sources, and comprises: database/data repository, text files, eXtensible Markup Language (XML) files, Message queues (IBM MQ) or Message services (Java Message Service, JMS)), and the like. The system can seamlessly integrate various independent data sources required by user decision, rapidly complete the deployment of a customized scoring model and a business rule strategy, and support the deployment environment of the business rule strategy to support the operations of If-then-else condition judgment, character string operation, various mathematical calculations and function formulas, For Loop cycle, decision matrix and the like; various decision application modules are constructed by combining a data model and a strategy rule; and batch processing and real-time processing are simultaneously supported. The first engine may be Vincio, which is not limited herein. Vincio supports database/data warehousing, text files, XML files, message queue (IBM MQ or JMS) data sources, and can acquire data to be processed through an input interface.
The policy rule file is a method file for processing and acquiring data to be processed. The policy rules file may be written in different kinds of programming languages, which may be XML. The obtained data to be processed are different, and the strategy rule files of the data to be processed are also different. For example, the policy rule document of the approval and quotation service may be adjusted according to the credit and income of the user for waiting for processing data; the strategy rule file of the transaction collection service can carry out transaction collection according to the user grade and the transaction time waiting processing data. For another example, in the image processing field, the image processing steps may be to perform image graying after denoising the image, perform binary differentiation on the grayed image, and perform morphological filtering on the image, and when the processed image is an image shot by a camera, perform processing such as denoising and then graying on the denoised image, and then perform binary differentiation and morphological filtering on the image; when the processed image is shot by the infrared shooting instrument, the image does not need to be subjected to graying processing, and the strategy rules for processing the image shot by the camera and the image shot by the infrared shooting instrument are different. The strategy rule file can have a regular structure, elements and attributes, the strategy rule file regular structure can be a father node, the father node comprises child nodes, the number of the child nodes is not limited, the child nodes can be used as new father nodes, and the new father nodes can comprise new child nodes; the rule content of the policy rule file is stored in the database through a rule structure, each hierarchical structure has hierarchical content, objects in the policy rule file appear in pairs, and the objects can be tags in the policy rule file. For example, a simple policy rules file for a bookstore:
<?xml version="1.0"encoding="utf-8"?>
<bookstore>
< | A! Recording book information >
Book Type, required course, ISBN, 7-111-
< title > data Structure </title >
< author > Xiaoming </author >
<price>30.00</price>
</book>
</bookstore>
In the policy rule file, a book is a parent node, a book is a child node of the book, and is a parent node of a title, a price and an author, the book and the attack exist in pairs, the book and the attack, the price and the attack are "essential lessons", "7-111-.
Specifically, the data to be processed is acquired through the first engine input interface, and the policy rule file in the first engine can be acquired according to the application scene characteristics in the data to be processed.
Step 206, converting the policy rule file in the first engine into a script file in the second engine; the first engine is used for writing the policy rule file, and the second engine is used for executing the policy rule file in the first engine.
The second engine is a strategy engine, which is used for executing the script file obtained after the strategy rule file in the first engine is converted, and supporting expression (Boolean combination), special mathematical formula calculation (high precision), syntax analysis, custom keywords, customized secondary script and the like. The second engine may be a QLExpress policy engine.
The script file is written by a programming language which can be identified by the second engine, the programming language supported by the script file and the strategy rule file is different, and the syntactic format is also different. The script file in the second engine and the policy rule file in the first engine may be the same as the method for acquiring the data to be processed, and the syntax and format of the file programming language are different.
Specifically, a policy rule file in the first engine is obtained, and the text parser may convert the policy rule file into a document object model object (DOM object) and a stream model object, and sequentially parse the DOM object into a script file that can be recognized by the second engine, starting from the element content corresponding to the parent node according to the rule structure of the policy rule file. The flow model object is an element of the flow calculation model for performing data processing on the flow, and the flow calculation model is a model for performing processing on the flow. For example, the text parser parses a policy rule file written in XML, the DOM parser converts the policy rule file into an XML DOM object, and then the XML DOM object is processed sequentially from the content of the element corresponding to the parent node to obtain a script file that can be recognized by the second engine.
And step 208, processing the data to be processed according to the script file through the second engine.
Specifically, the script file is a policy rule for recording a format that the second engine can recognize. The second engine calls the script file through a Dynamic Link Library (DLL), reads the data to be processed based on a data interface in the script file, and processes the data to be processed. The dynamic link library can be self-defined or self-contained in the system, and can be a mode for realizing the concept of the shared function library. For example: defining three integer variables a, b and c, a SUM function SUM (int a and int b) with parameters in a dynamic link library, wherein a function expression c is a + b, and a second engine is connected to a script file through an API (application programming interface) of a dynamic link library; the script file transmits the data A to be processed to 5 and the data B to 4 to the parameters a and B in the SUM (int a and int B), so that the summation of a and B in the script file is realized.
In the data processing method, the characteristic information of the data to be processed is acquired, the strategy rule file in the first engine corresponding to the data to be processed is acquired according to the characteristic information, and the corresponding strategy rule file is acquired from the strategy file library in the first engine according to the characteristic information of the data, so that the data acquisition time can be shortened; and converting the strategy rule file in the first engine into a script file in a second engine, wherein the first engine is used for writing the strategy rule file, the second engine is used for processing the data to be processed according to the script file, and the second engine is used for processing the data to be processed independently, so that the calculation performance of the data processing process is improved.
In one embodiment, obtaining the data to be processed comprises:
acquiring data to be processed through an input interface;
the data to be processed includes application environment data or includes application environment data and user data.
The input interface is an interface for importing input to the first engine. The input interface can select the corresponding interface according to different data types. The user data may be relevant information for the user, such as the user's company name, the user's credit report, and the like. The application environment data comprises an application environment type, and the application environment type can be a web site access on a webpage, a WeChat applet, a Payment application service and the like. And different data types select corresponding input interfaces, so that the accuracy of data and the data transmission speed are ensured.
In one embodiment, the obtaining the feature information of the data to be processed and the obtaining the policy rule file in the first engine corresponding to the data to be processed according to the feature information includes:
acquiring application environment data in the data to be processed;
and acquiring the environment type of the application environment data, and acquiring a policy rule file in the first engine corresponding to the data to be processed according to the environment type.
Specifically, the environment type can be a webpage, an APP, a WeChat applet and the like, the policy rule file in the first engine corresponding to the data to be processed is directly obtained through the environment type of the application environment data, the query time of the policy rule file is reduced, and the calculation performance is improved.
In one embodiment, converting the policy rules file in the first engine to a script file in the second engine comprises:
acquiring a version parameter of a first engine and a policy rule file identifier selected in a policy rule conversion interface;
acquiring a corresponding policy rule file according to the policy rule file identifier;
acquiring a corresponding file conversion rule according to the version parameter;
and converting the strategy rule file by adopting a file conversion rule to obtain a script file in the second engine.
Specifically, a first engine that selects a different version may be used for training comparison, evaluation and verification of the execution results of the policy rules file. For example, the processing result obtained by putting the online first engine version 1.0 and 20 policy rule files into actual production operation is 2200, the processing result obtained by verifying the online first engine version 2.0 and 35 policy rule files is 2500, the preset value fluctuation range is 2000 and 2800, and the accuracy of the processing result of the first engine version 1.0 is reliable according to the value fluctuation range. The policy rule file includes different policy rule files, each of which has a different identifier, for example, the policy rule file may include a policy rule file a, a policy rule file B, a policy rule file C, and the like. Therefore, training comparison is realized in a production environment, the strategy rule file is modified in time, and the reliability of data is improved.
In one embodiment, the file conversion rule is adopted to convert the policy rule file, and the script file in the second engine is obtained;
acquiring a file node structure of the strategy rule file;
and converting the strategy rule file in the first engine according to the file node structure to obtain a script file in a second engine.
The node structure may be a node tree, and may be used to represent the positional relationship of all nodes in the policy rule file. The top of the node tree may be the root node of the node structure, each node except the node may have a parent node, the node may have an indefinite number of child nodes, and the peer nodes may have the same parent node. And accessing the corresponding content of the node according to the node structure, and orderly converting the strategy rule file.
In one embodiment, processing the data to be processed according to the script file includes:
acquiring a component list in the script file, and acquiring components in the component list;
calling a component function corresponding to the component to process a corresponding component script file to obtain a script execution file;
and processing the data to be processed according to the script execution file.
The component list can include an indefinite number of components, the list in the component list can be sorted according to default 0,1,2,3 … …, and 0 represents the first component; the component can be an encapsulation package encapsulated by data and methods, and the component can be reused; a component function is a method for processing a corresponding component script. The corresponding component function processes the corresponding component script file, so that the efficiency and the accuracy of processing the component script are improved.
In one embodiment, calling a component function corresponding to the component to process a corresponding component script file, and obtaining a script execution file includes:
calling a component script file corresponding to the component function processing;
when the current component function finishes processing the corresponding component script file, acquiring the output arrow direction corresponding to the component;
when a corresponding arrow tail direction value is obtained according to the direction of an output arrow, calling a next component function in the current component function;
when the corresponding arrow tail direction value is not obtained according to the direction of the output arrow, jumping out of the current component function, and executing the next component function;
and when the component script files corresponding to the components in the component list are processed, obtaining script execution files.
Specifically, the arrow (cos) direction is associated with the component list order, the cos direction representing the order of the components in the component list; for example, there are 10 components in the component list, the components are numbered sequentially from 0 to 9, 0 represents one component, 9 represents the tenth component, and the direction of cos direction is the direction of processing the script files corresponding to the first component to the tenth component sequentially; for example, when the script file corresponding to the second component is processed after the script file corresponding to the first component is processed, the cos arrow points to the second component from the first component, and the arrow direction value of the cos arrow is 1, the component with the next component number of 1, that is, the second component, can be known through the mapping relationship. The component scripts can be processed in order to obtain script files, so that the response time of script execution is shortened, and the calculation performance is improved.
In one embodiment, as shown in fig. 3, a data processing overall design is provided, which is described by taking the method as an example applied to the terminal in fig. 1, and includes the following steps:
step 302, a policy rule file is obtained.
Specifically, the policy rule file may be a policy rule file written in the programming language for the data to be processed after logging in the first engine, where the policy rule file is written in a syntax and a syntax structure that conform to the programming language. For example, the first engine may be a Vincio decision engine, the policy rule file is written in the Vincio decision engine visualization management tool and the XML programming language, and if the policy rule file conforms to the XML language writing format and syntax, the policy rule file is valid, and the valid policy rule file is stored in the database.
And step 304, converting the written policy rule file into a script file.
Specifically, in a first engine interface policy rule file conversion interface, a conversion rule corresponding to the version parameter of the first engine and the identifier of the policy rule file is acquired, and the selected policy rule file is converted into a script file which can be recognized by a second engine. For example, the script file may be a QL script file recognizable by the QLExpress policy engine.
Step 306, the converted script file is analyzed.
Wherein, the script file can comprise a label element value, a component list, a component algorithm script and an arrow connection list; the component list includes an indefinite number of components and the arrow connection list may include an indefinite number of arrows. The script file after the analysis and conversion can obtain a corresponding global variable set and an input/output function according to the label element value of the script file. And calling the component function corresponding to the component in the component list in the dynamic link library, and processing the corresponding component script to obtain the text command for specifically realizing the component function.
Specifically, the parsing process for parsing the converted script file includes: establishing a global variable set, storing element values of elements in the script file in the global variable set, wherein the element values of the elements in the global variable set can be called by a corresponding function in the whole analysis process, for example, the global variable set is rukouMap, and the element values of widf and ponents elements can be stored in the rukouMap; establishing an input/output function, wherein the input/output function may call element values of elements in the global variable set as inputs, and process an output processing result through the input/output function, for example, the input/output function is function rukou _1(rukouMap), and the input/output function may call element values of widf in the global variable set, for example, rukouMap. widf. value ═ 1, and output a result; executing a component function acquiring operation according to an output result, that is, acquiring a component function corresponding to a component in the component list, where each component function defines a local variable, and the local variable is only applicable to the component function, for example, acquiring a component function loan _ pont (loanMap) of the component loanPonent, where the loanMap is a local variable of the component function and is only invoked in the loanPonent (loanMap) function; the input parameters of the component function can be the output parameters of the component function of the previous component, but the input parameters of the component function are given to initial values, and the initial values are covered when the output parameters of the component function of the previous component are received; the corresponding component script file is parsed by the component function, that is, the script file is parsed into the text command executable by the second engine, for example, the lonponent component script file is parsed by the lonjponent (lonmap), so as to obtain the text command executable by the second engine.
And 308, acquiring a script file calculation result.
Specifically, when the script file is executed, the script file is read through the dynamic link library according to a text command analyzed by the script file, and a calculation result is obtained.
For example, Object r is runner.
Express may be a text command that the component function can implement after the script parsing is finished, context may be band processing data such as user data and application environment data, null may be a log of no need to execute printing error information, True may be a set of instructions to cache itself, and false may be no need to output execution order information.
The data processing overall design is used for acquiring the characteristic information of the data to be processed and acquiring a strategy rule file in a first engine corresponding to the data to be processed according to the characteristic information; converting the policy rule file in the first engine into a script file in a second engine; the first engine is used for writing a policy rule file, and the second engine is used for executing the converted script file; and processing the data to be processed according to the script file. The second engine shortens the time for reading the script file, improves the calculation performance of data processing, realizes training contrast in a production environment after the strategy rule file is converted into the script file, can find problems in time and reduces risks caused by processing data by the script file.
It should be understood that although the various steps in the flow diagrams of fig. 2-3 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2-3 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternating with other steps or at least some of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 4, there is provided a policy rules file structure comprising tag elements: a policy rule file root directory 402, a policy rule file attribute 404, a component list 406, a component 408, an arrow connection list 410, an arrow connection 412; wherein:
the policy rule file root directory 402 is used for indicating a plurality of nodes of the policy rule file structure, and contains the description information and the rule content of the policy rule file.
The policy rule file attribute 404 is used to represent specific characteristics of the policy rule file, and distinguish different projects. For example, in the financial business, the attribute of the approval adjustment business is approval adjustment, and the attribute of the transaction collection business is transaction collection.
Component list 406, which represents a collection of components.
Component 408, which represents a package of data and methods.
The arrow connects to list 410 to represent a set of arrows.
The arrow connections 412 are used to describe the relationship description between the components, and may be dependencies, associations, inheritances, generalizations, aggregations, and the like.
In the above policy rule file structure, each tag element is paired, and the policy rule file root directory 402 of the policy rule file may include a policy rule file attribute 404, a component list 406, a component 408, an arrow connection list 410, and an arrow connection 412. The characteristic information of the data to be processed is obtained, the strategy rule file in the first engine corresponding to the data to be processed is obtained according to the characteristic information, the strategy rule file is converted into the script file in the second engine according to the strategy rule file structure, and the data to be processed is processed according to the script file, so that the calculation performance of the data processing process is improved.
In one embodiment, as shown in FIG. 5, a policy rules file transformation interface in the first engine is provided, including a first engine version option 502, a policy file option 504, a query button 506, a transformation button 508; wherein:
a first engine version option 502 for selecting a first engine version option version from the selectable boxes according to the actual production environment.
A policy file option 504, configured to select, from multiple engineering packages, an engineering package that meets the current service pending data, where the engineering package includes the description information of the engineering and the content of the policy rule file.
And querying 506 for the optional engineering package corresponding to the first engine version option.
And a conversion 508, configured to perform conversion according to the selected first engine version option and the corresponding policy rule file.
The version option version and the project package of the first engine are selected by self-definition, the strategy rule file in the first engine is converted into the script file in the second engine, and the data to be processed is processed according to the script file, so that the training comparison of production and application is realized, the reliability of the data is improved, and the calculation performance is improved.
In one embodiment, as shown in FIG. 6, there is provided a structure of a data processing apparatus 600 comprising: a data acquisition module 602, a policy rule file acquisition module 604, a file processing module 606, and a data processing module 608; wherein:
a data obtaining module 602, configured to obtain data to be processed.
The policy rule file obtaining module 604 is configured to obtain, according to the feature information of the data to be processed, a policy rule file in the first engine corresponding to the data to be processed.
And a file processing module 606 for converting the policy rule file in the first engine into a script file in the second engine.
And the data processing module 608 is configured to process the data to be processed according to the script file in the second engine.
The data processing device acquires the characteristic information of the data to be processed and acquires a strategy rule file in a first engine corresponding to the data to be processed according to the characteristic information; converting the policy rule file in the first engine into a script file in a second engine; the first engine is used for writing a policy rule file, and the second engine is used for converting the policy rule file in the first engine; and processing the data to be processed according to the script file. The second engine executes the script file faster than the first engine executes the strategy rule file, so that the calculation performance of data processing is improved, training comparison is realized in a production environment after the strategy rule file is converted into the script file, problems are found in time, and the risk caused by processing data by the script file is reduced. The division of each module in the above apparatus is only for illustration, and in other embodiments, the recommendation information generation apparatus may be divided into different modules as needed to complete all or part of the functions of the recommendation information generation apparatus.
In one embodiment, as shown in FIG. 6, the data processing apparatus 600 further comprises a parameter determination module 610; wherein:
a parameter determination module 610, configured to determine a version parameter and a policy rule file identification of the first engine.
In one embodiment, as shown in FIG. 6, the data processing apparatus 600 further comprises an extraction module 612; wherein:
and the extracting module 612 is configured to extract a file node structure of the policy rule file.
In one embodiment, as shown in fig. 6, the data processing apparatus 600 further comprises a component obtaining module 614, a function calling module 616; wherein:
and the component acquiring module 614 is used for acquiring the components in the component list in the script file.
And a function calling module 616, configured to call a component function corresponding to the component.
In one embodiment, as shown in fig. 6, the data processing apparatus 600 further comprises a determining module 618; wherein:
a determining module 618, configured to determine that, when the arrow direction is output and a corresponding arrow tail direction value is obtained, a next component function is called in the current component function;
and when the output arrow direction does not obtain the corresponding arrow tail direction value, jumping out of the current component function and executing the next component function.
The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 7. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program to run on the non-volatile storage medium. The database of the computer device is used for storing data to be processed. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a data processing method.
Those skilled in the art will appreciate that the architecture shown in fig. 7 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory having a computer program stored therein and a processor that when executing the computer program performs the steps of:
acquiring data to be processed;
acquiring characteristic information of data to be processed, and acquiring a strategy rule file in a first engine corresponding to the data to be processed according to the characteristic information;
converting the strategy rule file in the first engine into a script file in the second engine; the first engine is used for writing a policy rule file, and the second engine is used for executing the policy rule file in the first engine;
and processing the data to be processed according to the script file through the second engine.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
acquiring data to be processed through an input interface;
the data to be processed comprises application environment data or comprises application environment data and user data.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
acquiring application environment data in data to be processed;
and acquiring the environment type of the application environment data, and acquiring a policy rule file in the first engine corresponding to the data to be processed according to the environment type.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
acquiring the version parameter of the first engine and the policy rule file identifier selected in a policy rule conversion interface;
acquiring a corresponding policy rule file according to the policy rule file identifier;
acquiring a corresponding file conversion rule according to the version parameter;
and converting the strategy rule file by adopting the file conversion rule to obtain a script file in a second engine.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
acquiring a file node structure of the policy rule file;
and converting the strategy rule file in the first engine according to the file node structure to obtain a script file in a second engine.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
acquiring a component list in the script file, and acquiring components in the component list;
calling a component script file corresponding to the component function processing to obtain a script execution file;
and processing the data to be processed according to the script execution file.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
calling a component script file corresponding to the component function processing;
when the current component function finishes processing the corresponding component script file, acquiring the output arrow direction corresponding to the component;
when a corresponding arrow tail direction value is obtained according to the output arrow direction, calling a next component function in the current component function;
when the corresponding arrow tail direction value is not obtained according to the output arrow direction, jumping out of the current component function, and executing the next component function;
and when the component script files corresponding to the components in the component list are processed, obtaining script execution files.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring data to be processed;
acquiring characteristic information of the data to be processed, and acquiring a policy rule file in a first engine corresponding to the data to be processed according to the characteristic information;
converting the policy rule file in the first engine into a script file in a second engine; the first engine is used for writing a policy rule file, and the second engine is used for executing the policy rule file in the first engine;
and processing the data to be processed according to the script file through the second engine.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring the data to be processed through an input interface;
the data to be processed comprises application environment data or comprises application environment data and user data.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring application environment data in the data to be processed;
and acquiring the environment type of the application environment data, and acquiring a policy rule file in the first engine corresponding to the data to be processed according to the environment type.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring the version parameter of the first engine and the policy rule file identifier selected in a policy rule conversion interface;
acquiring a corresponding policy rule file according to the policy rule file identifier;
acquiring a corresponding file conversion rule according to the version parameter;
and converting the strategy rule file by adopting the file conversion rule to obtain a script file in a second engine.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring a file node structure of the policy rule file;
and converting the strategy rule file in the first engine according to the file node structure to obtain a script file in a second engine.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring a component list in the script file, and acquiring components in the component list;
calling a component script file corresponding to the component function processing to obtain a script execution file;
and processing the data to be processed according to the script execution file.
In one embodiment, the computer program when executed by the processor further performs the steps of:
calling a component script file corresponding to the component function processing;
when the current component function finishes processing the corresponding component script file, acquiring the output arrow direction corresponding to the component;
when a corresponding arrow tail direction value is obtained according to the output arrow direction, calling a next component function in the current component function;
when the corresponding arrow tail direction value is not obtained according to the output arrow direction, jumping out of the current component function, and executing the next component function;
and when the component script files corresponding to the components in the component list are processed, obtaining script execution files.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above may be implemented by hardware instructions of a computer program, which may be stored in a non-volatile computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (10)

1. A method of data processing, the method comprising:
acquiring data to be processed;
acquiring characteristic information of the data to be processed, and acquiring a policy rule file in a first engine corresponding to the data to be processed according to the characteristic information;
converting the policy rule file in the first engine into a script file in a second engine; the first engine is a decision engine and is used for compiling a strategy rule file, training and contrasting, and evaluating and verifying the execution result of the strategy rule file; the second engine is a policy engine and is used for executing the policy rule file in the first engine; the script file and the strategy rule file support different programming languages and different syntactic formats;
acquiring a component list in the script file, and acquiring components in the component list;
calling a component script file corresponding to the component function processing;
when the current component function finishes processing the corresponding component script file, acquiring the output arrow direction corresponding to the component;
when a corresponding arrow tail direction value is obtained according to the output arrow direction, calling a next component function in the current component function; the arrow direction is associated with the component list order and represents the order of the components in the component list; the arrow tail direction value has a mapping relation with the components in the component list;
when the corresponding arrow tail direction value is not obtained according to the output arrow direction, jumping out of the current component function, and executing the next component function;
when the component script files corresponding to the components in the component list are processed, obtaining script execution files;
and processing the data to be processed according to the script execution file.
2. The method of claim 1, wherein the obtaining the data to be processed comprises:
acquiring the data to be processed through an input interface;
the data to be processed comprises application environment data or comprises application environment data and user data.
3. The method according to claim 1, wherein the obtaining feature information of the data to be processed and obtaining the policy rule file in the first engine corresponding to the data to be processed according to the feature information comprises:
acquiring application environment data in the data to be processed;
and acquiring the environment type of the application environment data, and acquiring a policy rule file in the first engine corresponding to the data to be processed according to the environment type.
4. The method of claim 1, wherein converting the policy rules file in the first engine to a script file in a second engine comprises:
acquiring the version parameter of the first engine and the policy rule file identifier selected in a policy rule conversion interface;
acquiring a corresponding policy rule file according to the policy rule file identifier;
acquiring a corresponding file conversion rule according to the version parameter;
and converting the strategy rule file by adopting the file conversion rule to obtain a script file in a second engine.
5. The method of claim 4, wherein the converting the policy rule file using the file conversion rule to obtain the script file in the second engine comprises;
acquiring a file node structure of the policy rule file;
and converting the strategy rule file in the first engine according to the file node structure to obtain a script file in a second engine.
6. A data processing apparatus, characterized in that the apparatus comprises:
the data acquisition module is used for acquiring data to be processed;
the policy rule file acquisition module is used for acquiring a policy rule file in a first engine corresponding to the data to be processed according to the characteristic information of the data to be processed;
the file processing module is used for converting the strategy rule file in the first engine into a script file in a second engine; the first engine is a decision engine and is used for compiling a strategy rule file, training and contrasting, and evaluating and verifying the execution result of the strategy rule file; the second engine is a policy engine and is used for executing the policy rule file in the first engine; the script file and the strategy rule file support different programming languages and different syntactic formats;
the component acquisition module is used for acquiring a component list in the script file and acquiring components in the component list;
the function calling module is used for calling a component script file corresponding to the component function processing;
the judging module is used for acquiring the output arrow direction corresponding to the component when the current component function finishes processing the corresponding component script file;
when a corresponding arrow tail direction value is obtained according to the output arrow direction, calling a next component function in the current component function; the arrow direction is associated with the component list order and represents the order of the components in the component list; the arrow tail direction value has a mapping relation with the components in the component list;
when the corresponding arrow tail direction value is not obtained according to the output arrow direction, jumping out of the current component function, and executing the next component function;
the data processing module is used for obtaining a script execution file when the component script file corresponding to each component in the component list is processed;
and processing the data to be processed according to the script execution file.
7. The apparatus of claim 6, wherein the data obtaining module is further configured to obtain the data to be processed through an input interface; the data to be processed comprises application environment data or comprises application environment data and user data.
8. The apparatus according to claim 6, wherein the policy rule file obtaining module is further configured to obtain application environment data in the data to be processed; and acquiring the environment type of the application environment data, and acquiring a policy rule file in the first engine corresponding to the data to be processed according to the environment type.
9. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 5 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 5.
CN201910614641.9A 2019-07-09 2019-07-09 Data processing method, data processing device, computer equipment and storage medium Active CN110502483B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910614641.9A CN110502483B (en) 2019-07-09 2019-07-09 Data processing method, data processing device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910614641.9A CN110502483B (en) 2019-07-09 2019-07-09 Data processing method, data processing device, computer equipment and storage medium

Publications (2)

Publication Number Publication Date
CN110502483A CN110502483A (en) 2019-11-26
CN110502483B true CN110502483B (en) 2022-06-17

Family

ID=68586213

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910614641.9A Active CN110502483B (en) 2019-07-09 2019-07-09 Data processing method, data processing device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN110502483B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113450463B (en) * 2021-07-01 2023-07-21 网易(杭州)网络有限公司 Preform conversion method and apparatus

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103440128B (en) * 2013-08-21 2016-08-10 交通银行股份有限公司 Data processing method and device
EP3483739A1 (en) * 2017-05-12 2019-05-15 QlikTech International AB Cognitive rule engine
CN107590186B (en) * 2017-08-07 2021-07-30 北京京东尚科信息技术有限公司 Method for managing and executing data processing policy and policy engine system
CN108428103B (en) * 2017-12-28 2020-03-10 深圳萨摩耶互联网金融服务有限公司 Decision engine and decision method
CN109766124B (en) * 2018-12-26 2022-05-03 深圳左邻永佳科技有限公司 Service development method, device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN110502483A (en) 2019-11-26

Similar Documents

Publication Publication Date Title
CN110688307B (en) JavaScript code detection method, device, equipment and storage medium
CN111352907A (en) Method and device for analyzing pipeline file, computer equipment and storage medium
CN112015430A (en) JavaScript code translation method and device, computer equipment and storage medium
CN109308254B (en) Test method, test device and test equipment
CN110020358B (en) Method and device for generating dynamic page
CN108090351A (en) For handling the method and apparatus of request message
CN112115338A (en) Page query request processing method, device, equipment and storage medium
CN114817968B (en) Method, device and equipment for tracing path of featureless data and storage medium
CN105678127A (en) Verification method and device for identity information
CN111783415B (en) Template configuration method and device
JP4951416B2 (en) Program verification method and program verification apparatus
CN113918794B (en) Enterprise network public opinion benefit analysis method, system, electronic equipment and storage medium
CN115599386A (en) Code generation method, device, equipment and storage medium
CN113535587B (en) Target application detection method and device and computer equipment
CN107368500B (en) Data extraction method and system
CN114398138A (en) Interface generation method and device, computer equipment and storage medium
CN110502483B (en) Data processing method, data processing device, computer equipment and storage medium
CN116340989A (en) Data desensitization method and device, electronic equipment and storage medium
CN113360300B (en) Interface call link generation method, device, equipment and readable storage medium
CN113435950B (en) Bill processing method and device
CN114491528A (en) Malicious software detection method, device and equipment
EP3460677A1 (en) Assessment program, assessment device, and assessment method
CN111475403A (en) Dynamic generation method of test script and related device
CN112650600B (en) Method, device and computer equipment for pushing message content
CN111310414B (en) RDF format file analysis method and generation method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CP03 Change of name, title or address

Address after: 518000 Room 201, building A, No. 1, Qian Wan Road, Qianhai Shenzhen Hong Kong cooperation zone, Shenzhen, Guangdong (Shenzhen Qianhai business secretary Co., Ltd.)

Patentee after: Zhaolian Consumer Finance Co.,Ltd.

Country or region after: China

Address before: 518000 Room 201, building A, No. 1, Qian Wan Road, Qianhai Shenzhen Hong Kong cooperation zone, Shenzhen, Guangdong (Shenzhen Qianhai business secretary Co., Ltd.)

Patentee before: MERCHANTS UNION CONSUMER FINANCE Co.,Ltd.

Country or region before: China

CP03 Change of name, title or address