CN112015636A

CN112015636A - Decision engine testing method and device based on support vector machine and electronic equipment

Info

Publication number: CN112015636A
Application number: CN202010677993.1A
Authority: CN
Inventors: 李林; 南冰
Original assignee: Beijing Qilu Information Technology Co Ltd
Current assignee: Beijing Qilu Information Technology Co Ltd
Priority date: 2020-07-14
Filing date: 2020-07-14
Publication date: 2020-12-01

Abstract

The invention discloses a decision engine testing method and device based on a support vector machine and electronic equipment, wherein the method comprises the following steps: generating a test sample set; executing a plurality of test strategies of the test sample set and outputting a plurality of test results; and analyzing the plurality of test results to generate a test evaluation report. The invention can automatically generate a test sample set, and output a plurality of test results by executing a plurality of test strategies of the test sample set; analyzing the plurality of test results may automatically generate a test evaluation report. The invention can automatically generate the test sample set and the test strategy evaluation result, thereby reducing the test cost of the decision engine and improving the test efficiency.

Description

Decision engine testing method and device based on support vector machine and electronic equipment

Technical Field

The invention relates to the technical field of computers, in particular to a decision engine testing method and device based on a support vector machine, electronic equipment and a computer readable medium.

Background

Due to the quality (such as the availability and stability of each function) of decision engines (such as a wind control decision engine, a marketing decision engine, a credit decision engine and the like), the method plays a crucial role in judging whether the decision result is correct or not. Therefore, before applying a decision engine to a production environment, functions of the decision engine need to be tested completely and rigorously, and after determining that each function of the decision engine has availability and stability, the decision engine is applied to the production environment.

In the existing decision engine test mode, taking a wind control decision engine as an example, business personnel need a full-combination coverage test in a wind control rule test, and the test cost is relatively high; during policy evaluation, service personnel need to comprehensively judge the policy result in multiple dimensions, and the policy evaluation efficiency is low. Therefore, it is urgently needed to improve rule testing efficiency, quickly obtain a policy evaluation result and reduce rule testing and policy evaluation pressure of business personnel on the premise of ensuring testing quality.

Disclosure of Invention

The invention aims to solve the technical problems of high test cost and low test efficiency of the conventional decision engine.

In order to solve the above technical problem, a first aspect of the present invention provides a decision engine testing method based on a support vector machine, where the method includes:

generating a test sample set;

executing a plurality of test strategies of the test sample set and outputting a plurality of test results;

and analyzing the plurality of test results to generate a test evaluation report.

According to a preferred embodiment of the present invention, the generating the test sample set includes:

creating a test set generation model based on a support vector machine method;

and generating a test sample set according to the test set generation model.

creating a test sample library;

selecting a test sample set from the test sample library.

According to a preferred embodiment of the present invention, before executing the plurality of test strategies of the test sample set, the method further comprises:

creating a test strategy set library;

and selecting a plurality of test strategies from the test strategy set library.

According to a preferred embodiment of the present invention, the analyzing the plurality of test results to generate a test evaluation report includes:

comparing the plurality of test results, and generating a key index comparison result according to the configured key index;

and generating a test strategy evaluation result according to the comparison result of the key indexes.

According to a preferred embodiment of the present invention, before generating the key index comparison result according to the configured key indexes, the method further includes:

creating an analysis method index library;

and selecting configured key indexes from the index library of the analysis method.

According to a preferred embodiment of the present invention, the analysis method index library includes configuration key indexes generated by two analysis methods, i.e., analysis method-function test and analysis index-pass rate.

In order to solve the above technical problem, a second aspect of the present invention provides a decision engine testing apparatus based on a support vector machine, the apparatus comprising:

the generating module is used for generating a test sample set;

the test module is used for executing a plurality of test strategies of the test sample set and outputting a plurality of test results;

and the analysis and evaluation module is used for analyzing the plurality of test results to generate a test and evaluation report.

According to a preferred embodiment of the present invention, the generating module includes:

the first generation module is used for creating a test set generation model based on a support vector machine method;

and the second generation module is used for generating a test sample set according to the test set generation model.

the first establishing module is used for establishing a test sample library;

and the first selection module is used for selecting a test sample set from the test sample library.

According to a preferred embodiment of the invention, the device further comprises:

the second establishing module is used for establishing a test strategy set library;

and the second selection module is used for selecting a plurality of test strategies from the test strategy collection library.

According to a preferred embodiment of the invention, the analytical evaluation module comprises:

the comparison generation module is used for comparing the plurality of test results and generating a key index comparison result according to the configured key index;

and the sub-generation module is used for generating a test strategy evaluation result according to the key index comparison result.

a third establishing module, configured to establish an analysis method index library;

and the third selection module is used for selecting the configured key indexes from the index library of the analysis method.

To solve the above technical problem, a third aspect of the present invention provides an electronic device, comprising:

a processor; and

a memory storing computer executable instructions that, when executed, cause the processor to perform the method described above.

In order to solve the above technical problem, a fourth aspect of the present invention proposes a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs that, when executed by a processor, implement the above method.

The invention can automatically generate a test sample set, and output a plurality of test results by executing a plurality of test strategies of the test sample set; analyzing the plurality of test results may automatically generate a test evaluation report. The invention can automatically generate the test sample set and the test strategy evaluation result, thereby reducing the test cost of the decision engine and improving the test efficiency.

Drawings

In order to make the technical problems solved by the present invention, the technical means adopted and the technical effects obtained more clear, the following will describe in detail the embodiments of the present invention with reference to the accompanying drawings. It should be noted, however, that the drawings described below are only illustrations of exemplary embodiments of the invention, from which other embodiments can be derived by those skilled in the art without inventive step.

FIG. 1 is a flow chart of a decision engine testing method based on a support vector machine according to the present invention;

FIG. 2 is a flow chart of another decision engine testing method based on a support vector machine according to the present invention;

FIG. 3 is a schematic diagram of the present invention creating a test strategy set library and an analysis method index library;

FIG. 4 is a schematic structural framework diagram of a decision engine testing apparatus based on a support vector machine according to the present invention;

FIG. 5 is a block diagram of an exemplary embodiment of an electronic device in accordance with the present invention;

FIG. 6 is a diagrammatic representation of one embodiment of a computer-readable medium of the present invention.

Detailed Description

Exemplary embodiments of the present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which exemplary embodiments of the invention may be embodied in many specific forms, and should not be construed as limited to the embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the invention to those skilled in the art.

The structures, properties, effects or other characteristics described in a certain embodiment may be combined in any suitable manner in one or more other embodiments, while still complying with the technical idea of the invention.

In describing particular embodiments, specific details of structures, properties, effects, or other features are set forth in order to provide a thorough understanding of the embodiments by one skilled in the art. However, it is not excluded that a person skilled in the art may implement the invention in a specific case without the above-described structures, performances, effects or other features.

The flow chart in the drawings is only an exemplary flow demonstration, and does not represent that all the contents, operations and steps in the flow chart are necessarily included in the scheme of the invention, nor does it represent that the execution is necessarily performed in the order shown in the drawings. For example, some operations/steps in the flowcharts may be divided, some operations/steps may be combined or partially combined, and the like, and the execution order shown in the flowcharts may be changed according to actual situations without departing from the gist of the present invention.

The block diagrams in the figures generally represent functional entities and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.

The same reference numerals denote the same or similar elements, components, or parts throughout the drawings, and thus, a repetitive description thereof may be omitted hereinafter. It will be further understood that, although the terms first, second, third, etc. may be used herein to describe various elements, components, or sections, these elements, components, or sections should not be limited by these terms. That is, these phrases are used only to distinguish one from another. For example, a first device may also be referred to as a second device without departing from the spirit of the present invention. Furthermore, the term "and/or", "and/or" is intended to include all combinations of any one or more of the listed items.

Referring to fig. 1, fig. 1 is a flowchart of a decision engine testing method based on a support vector machine according to the present invention, as shown in fig. 1, the method includes:

s1, generating a test sample set;

in one mode, a test set generation model can be created based on a support vector machine method, and then the test set generation model is used for generating a test sample set with moderate scale and high coverage. Support Vector Machines (SVMs) map vectors into a higher dimensional space in which a maximally spaced hyperplane is established. Two hyperplanes are built parallel to each other on both sides of the hyperplane separating the data. Establishing a suitably oriented separation hyperplane maximizes the distance between two hyperplanes parallel thereto. It is assumed that the larger the distance or difference between the parallel hyperplanes, the smaller the total error of the classifier. The support vector machine is similar to a neural network and is a learning mechanism, but is different from the neural network in that a mathematical method and an optimization technology are used by the SVM, so that the support vector machine has a plurality of specific advantages in the process of solving the recognition of small samples, non-linear and high-dimensional patterns, and can be popularized and applied to other machine learning problems such as function fitting.

In another approach, the test sample library is created; and selecting a test sample set from the test sample library according to a user instruction. Wherein the test sample library may be generated by collecting production data.

S2, executing a plurality of test strategies of the test sample set and outputting a plurality of test results;

the plurality of test strategies may include a production strategy, a quasi-production strategy, and a historical strategy, among others. Specifically for the financial industry, the plurality of test policies may include online policies and offline policies as well as historical policies. After executing a plurality of test strategies, the same test sample set outputs a plurality of corresponding test results.

And S3, analyzing the plurality of test results to generate a test evaluation report.

Specifically, the method and the device generate a key index comparison result according to the pre-configured key index by comparing a plurality of test results. And evaluating the influence of the strategy according to the change condition of each key index, and generating a test evaluation report.

Fig. 2 is a flowchart of another decision engine testing method based on a support vector machine according to the present invention, as shown in fig. 2, the method includes:

s201, creating a test set generation model based on a support vector machine method, and generating a test sample set according to the test set generation model.

Step S201 is the same as step S1, and is not described here.

S202, respectively creating a test strategy set library and an analysis method index library;

as shown in fig. 3, the test policy set library is used to store pre-generated policies, which may specifically include production policies, quasi-production policies, and historical policies. The strategies are generated according to the existing application scene data, each strategy corresponds to one application scene, and the strategy engine performs rule settlement for each strategy. In the simple rule, the variables are judged under various conditions and then concluded. Variables may include functions, input parameters, output parameters, temporary parameters. Conditional decisions can be implemented by logical operations, such as the introduction of operators and operators. The conclusion can be reached by the condition judgment of the variable, and the conclusion is the output of the rule. In the present invention, taking the financial industry as an example, the test policy set library may store online policies, offline policies, and historical policies.

The analysis method index library comprises configuration key indexes generated by two analysis modes of an analysis method, namely a function test and an analysis index, namely a passing rate. The analysis method-function test is an analysis mode for achieving a certain function test through an analysis method, and the analysis index-passing rate is an analysis mode for reflecting the passing rate by analyzing a certain index.

S203, selecting a plurality of test strategies from the test strategy collection library.

Different test strategies can be selected according to different test scene requirements. For example, for a test sample set and a test scenario that have already been determined, since different strategies correspond to different test scenarios, a plurality of test strategies under the test scenario may be selected according to the test scenario. As shown in fig. 3, in the same test scenario, a test strategy one, a test strategy two, and a test strategy three are selected.

S204, executing a plurality of test strategies of the test sample set, and outputting a plurality of test results;

in the invention, the test strategy can call one or a plurality of rules according to the requirements of the reference scene, and each rule judges the condition of the respective variable and then obtains the conclusion. Each rule calls functions and parameters needed to be used in a function library and a parameter library respectively, judgment is carried out according to the conditions of each rule, and then a conclusion is obtained. And the policy engine performs rule settlement on each rule and conclusion to obtain the final output of the policy. Wherein, different test strategies correspondingly output different test results. As shown in fig. 3, the first test strategy outputs the first test result, the second test strategy outputs the second test result, and the third test strategy outputs the third test result.

S205, selecting configured key indexes from the index library of the analysis method.

For example, configured key indexes can be selected from the analysis method index library according to user operation. Specifically, all configuration key indexes in the analysis method index library may be displayed first, and then the selected configuration key index may be determined according to a selection operation of a user, such as a click operation. As shown in FIG. 3, a configuration key indicator one is selected.

S206, comparing the plurality of test results, and generating a key index comparison result according to the configured key indexes;

as shown in fig. 3, the size, variation trend, etc. of the first configured key index in the multiple test results may be compared, so as to generate a key index comparison result.

And S207, generating a test strategy evaluation result according to the key index comparison result.

For example, the test strategy may be evaluated according to comparison of the variation trends of the key indexes, and a test strategy evaluation result may be generated.

Fig. 4 is a schematic diagram of an architecture of a decision engine testing apparatus based on a support vector machine according to the present invention, as shown in fig. 4, the apparatus includes:

a generating module 41, configured to generate a test sample set;

a testing module 42, configured to execute a plurality of testing strategies of the testing sample set and output a plurality of testing results;

and an analysis and evaluation module 43 for analyzing the plurality of test results to generate a test and evaluation report.

In one example, the generating module 41 includes:

In another example, the generating module 42 includes:

the first establishing module is used for establishing a test sample library;

The analysis evaluation module 43 includes:

Further, the apparatus further comprises:

A third establishing module, configured to establish an analysis method index library; the analysis method index library comprises configuration key indexes generated by two analysis modes of an analysis method, namely a function test mode, an analysis index mode and a passing rate mode.

Those skilled in the art will appreciate that the modules in the above-described embodiments of the apparatus may be distributed as described in the apparatus, and may be correspondingly modified and distributed in one or more apparatuses other than the above-described embodiments. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.

In the following, embodiments of the electronic device of the present invention are described, which may be regarded as an implementation in physical form for the above-described embodiments of the method and apparatus of the present invention. Details described in the embodiments of the electronic device of the invention should be considered supplementary to the embodiments of the method or apparatus described above; for details which are not disclosed in embodiments of the electronic device of the invention, reference may be made to the above-described embodiments of the method or the apparatus.

Fig. 5 is a block diagram of an exemplary embodiment of an electronic device according to the present invention. The electronic device shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.

As shown in fig. 5, the electronic device 500 of the exemplary embodiment is represented in the form of a general-purpose data processing device. The components of the electronic device 500 may include, but are not limited to: at least one processing unit 510, at least one memory unit 520, a bus 530 connecting different electronic device components (including the memory unit 520 and the processing unit 510), a display unit 540, and the like.

The storage unit 520 stores a computer readable program, which may be a code of a source program or a read-only program. The program may be executed by the processing unit 510 such that the processing unit 510 performs the steps of various embodiments of the present invention. For example, the processing unit 510 may perform the steps as shown in fig. 1.

The memory unit 520 may include a readable medium in the form of a volatile memory unit, such as a random access memory unit (RAM)5201 and/or a cache memory unit 5202, and may further include a read only memory unit (ROM) 5203. The memory unit 520 may also include a program/utility 5204 having a set (at least one) of program modules 5205, such program modules 5205 including, but not limited to: operating the electronic device, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

Bus 530 may be one or more of any of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

The electronic device 500 may also communicate with one or more external devices 300 (e.g., keyboard, display, network device, bluetooth device, etc.), enable a user to interact with the electronic device 500 via the external devices 500, and/or enable the electronic device 500 to communicate with one or more other data processing devices (e.g., router, modem, etc.). Such communication can occur via input/output (I/O) interfaces 550, and can also occur via network adapter 560 to one or more networks, such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet. The network adapter 560 may communicate with other modules of the electronic device 500 via the bus 530. It should be appreciated that although not shown in FIG. 5, other hardware and/or software modules may be used in the electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID electronics, tape drives, and data backup storage electronics, among others.

FIG. 6 is a schematic diagram of one computer-readable medium embodiment of the present invention. As shown in fig. 6, the computer program may be stored on one or more computer readable media. The computer readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electronic device, apparatus, or device that is electronic, magnetic, optical, electromagnetic, infrared, or semiconductor, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. The computer program, when executed by one or more data processing devices, enables the computer-readable medium to implement the above-described method of the invention, namely: generating a test sample set; executing a plurality of test strategies of the test sample set and outputting a plurality of test results; and analyzing the plurality of test results to generate a test evaluation report.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments of the present invention described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to make a data processing device (which can be a personal computer, a server, or a network device, etc.) execute the above-mentioned method according to the present invention.

The computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution electronic device, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including object oriented programming languages such as Java, C + + or the like and conventional procedural programming languages, such as "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

In summary, the present invention can be implemented as a method, an apparatus, an electronic device, or a computer-readable medium executing a computer program. Some or all of the functions of the present invention may be implemented in practice using a general purpose data processing device such as a microprocessor or a Digital Signal Processor (DSP).

While the foregoing embodiments have described the objects, aspects and advantages of the present invention in further detail, it should be understood that the present invention is not inherently related to any particular computer, virtual machine or electronic device, and various general-purpose machines may be used to implement the present invention. The invention is not to be considered as limited to the specific embodiments thereof, but is to be understood as being modified in all respects, all changes and equivalents that come within the spirit and scope of the invention.

Claims

1. A decision engine testing method based on a support vector machine is characterized by comprising the following steps:

generating a test sample set;

2. The method of claim 1, wherein the generating a set of test samples comprises:

creating a test set generation model based on a support vector machine method;

and generating a test sample set according to the test set generation model.

3. The method of any of claims 1-2, wherein the generating a set of test samples comprises:

creating a test sample library;

selecting a test sample set from the test sample library.

4. The method of any of claims 1-3, wherein prior to said executing the plurality of test strategies for the test sample set, the method further comprises:

creating a test strategy set library;

5. The method of any one of claims 1-4, wherein the analyzing the plurality of test results to generate a test assessment report comprises:

6. The method according to any one of claims 1-5, wherein before generating a key indicator comparison result from the configured key indicators, the method further comprises:

creating an analysis method index library;

7. The method according to any one of claims 1 to 6, wherein the analysis method index library comprises configuration key indexes generated by two analysis modes of analysis method-function test and analysis index-passing rate.

8. A decision engine testing apparatus based on a support vector machine, the apparatus comprising:

the generating module is used for generating a test sample set;

9. An electronic device, comprising:

a processor; and

a memory storing computer-executable instructions that, when executed, cause the processor to perform the method of any of claims 1-7.

10. A computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement the method of any of claims 1-7.