CN117785982A - Multi-data source online analysis test method under credit scene - Google Patents
Multi-data source online analysis test method under credit scene Download PDFInfo
- Publication number
- CN117785982A CN117785982A CN202410017928.4A CN202410017928A CN117785982A CN 117785982 A CN117785982 A CN 117785982A CN 202410017928 A CN202410017928 A CN 202410017928A CN 117785982 A CN117785982 A CN 117785982A
- Authority
- CN
- China
- Prior art keywords
- data source
- decision
- external
- scene
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 15
- 238000010998 test method Methods 0.000 title claims abstract description 6
- 238000000034 method Methods 0.000 claims abstract description 13
- 238000012360 testing method Methods 0.000 claims description 9
- 238000012795 verification Methods 0.000 claims description 8
- 230000003993 interaction Effects 0.000 claims description 5
- 210000001061 forehead Anatomy 0.000 claims description 3
- 230000002688 persistence Effects 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 description 6
- 238000011835 investigation Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013524 data verification Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012797 qualification Methods 0.000 description 1
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a multi-data source online analysis test method in a credit scene, which comprises the following steps: s1: establishing a unified external data debugging interface; s2: when a user initiates a transaction, calling is additionally carried out according to the configured scene set; s3: a called decision flow; s4: according to different scene codes, the method walks to different decision stream branches; s5: in each different scene branch, firstly obtaining the configuration of the number control of the sample to be debugged under the scene; s6: invoking a data source node in the decision flow; s7: the data source returns corresponding query results, the results are assigned to the decision flow, and rule configuration is carried out according to the need; s8: and asynchronously dropping the query result and the intermediate variable data through the message queue. The invention uniformly manages the data sources which need to be debugged on line, and can rapidly count which data sources exist, thereby improving the convenience in the debugging stage.
Description
Technical Field
The invention belongs to the technical field of online data verification, and particularly relates to an online analysis and test method for multiple data sources under a credit scene.
Background
According to relevant legal requirements such as "China people banking Law" China people Country, personal information protection Law, financial institutions cannot conduct business cooperation with market institutions which do not acquire legal credit investigation service qualification to acquire credit investigation service. Therefore, when a financial credit service is developed, if a new manufacturer number source is needed, the new manufacturer number source is accessed through a credit investigation mechanism, so that each time one number source is added, debugging and online verification are needed for the manufacturer and the credit investigation mechanism, the online verification is needed to meet a certain production sample number, and a database stores enough number source data samples.
At present, the existing data source access is to embed a data source module in each production real-time decision stream of a platform, each decision stream corresponds to the own external number to be debugged, when the real-time feed flow of each channel comes, the decision stream corresponding to the channel scene is called, and after the external data source is called in the decision stream in real time, the feed data is stored in a lifting library. But this approach has the following problems:
1. the data sources which need to be debugged online are not uniformly managed, and the data sources which need to be debugged online can be counted only by turning the data sources in the decision flow of the system;
2. the debugging data source is coupled with the production real-time decision-making feeding business, and when the debugging data source fails, the production real-time feeding is also affected;
3. the consumption of the data source is monitored and lost, so that the data cost is easily caused to be hyperbranched;
4. the configuration lacks flexibility, and the debugging data source is complicated in online and offline operation.
Disclosure of Invention
The invention mainly solves the technical problems that the prior proposal does not uniformly manage the data source which needs to be debugged on line, and simultaneously, the data source is debugged and the business of the real-time decision-making in-production is coupled together, which is easy to cause the data cost to be hyperbranched, the configuration lacks flexibility, and the debugging data source is complicated to be operated on line and off line.
In order to solve the technical problems, the invention adopts a technical scheme that:
a multi-data source online analysis test method under credit scene comprises the following steps:
s1): establishing a unified external data debugging interface: an interface capable of uniformly calling a fixed decision stream is newly added in a Controller module of the wind control system, an external data source access request needing to be debugged is received, and the external data source access request is forwarded to a corresponding countermeasure stream interface;
s2): when a user initiates a transaction, calling is additionally carried out according to the configured scene set: in credit transaction, a user initiates a request at a client, sends a self-executed decision flow interface request to a front-end system for verification through a front-end system, and simultaneously calls a unified online data source debugging decision flow interface;
s3): decision flow of call: exposing an online data source debugging decision flow interface, and receiving a transmitted request parameter into a decision flow, wherein an application module is an engine module, and the Controller module calls and receives the transmitted request parameter into the decision flow;
s4): according to different scene codes, branching to different decision streams: the unified online data source debugging decision flow is configured with different branches according to different scene codes, the request parameters sent by the Controller are provided with businesscodes, and the decision flow enters different business branch flows according to the corresponding businesscodes and flows under the current branches;
s5): in each different scene branch, serially acquiring configuration of the number control of the samples to be debugged in the scene through a system flow;
s6): invoking the required data source node at the decision flow: the external data source is called through the HTTP of the system, and in the flow of the data source debugging, the selected data source is called only through the external data calling module;
s7): the data source returns corresponding query results, the results are assigned to the decision flow, and rule configuration is carried out according to the need;
s8): asynchronous database-falling query results and intermediate variable data through a message queue: after the related foreign number debugging information is queried from the DSM, a database interaction module adopted by a consumer side is a DBdata module through a message queue, and after interaction with an Oracle through a persistence component mybatis, the fields to be stored are stored in a structured data table of the Oracle.
Furthermore, in the step S1, the debug interface may also be provided for the application internal engine module to call, and in the decision flow, the debug interface may be called to debug the external data that has not been verified yet.
Further, in the step S2, the verification of the front-end system includes identity information and data validity.
Further, in the step S4, the bussiness code is divided into the following branches:
channel 1: trust;
channel 2: letter use;
channel 3: and (5) adjusting the forehead.
Further, in the step S5, after entering the branch flow corresponding to the businessCode, the first node is the configuration of the number of samples and the expiration time of the external number source required to be debugged by the branch, and when the node is reached, the number of samples and the expiration time and the batch information required to be configured at this time can be obtained.
Further, in the step S7, when the execution is performed to the external debug data source call node, the intelligent decision engine may call the thirdData module, and when the configuration is configured as a plurality of data sources, the total time length, that is, the data source with the most time consumption, is called concurrently through the thread pool.
Further, the destination address of the external number call is a unified DSM, after the DSM receives the request of the decision engine, the DSM routes the request to different external number interfaces according to the apiCode, and then requests the external number manufacturer interface according to the transmitted client information, so as to obtain interface service information provided by the external number manufacturer in real time, and assign the information to the external number unified format and return the information, wherein the information also comprises sample number configuration information transmitted in the thirdData module.
Further, in the step S8, the producer is a decision engine, and the message body transmitted includes the information of the external data source for debugging the query and the intermediate variables generated in the decision stream.
Further, in the step S8, the table fields are defined by the application component, and then the database table is created for structured storage.
The invention has the beneficial effects that:
the invention carries out unified management on the data sources needing online debugging, can rapidly count which data sources appear, improves convenience in the debugging stage, changes the debugging data sources and the real-time production decision-making streaming business into branch processing, and does not influence other data when any one of the data sources has a problem.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a system module call flow provided by the present invention;
FIG. 2 is a schematic diagram of an online data source debug decision flow provided by the present invention;
Detailed Description
The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making clear and defining the scope of the present invention.
Specific example 1:
as shown in fig. 1 to 2, a method for testing multiple data sources in a credit scene by online analysis includes the following steps:
s1): establishing a unified external data debugging interface: an interface capable of uniformly calling a fixed decision stream is newly added in a Controller module (a function of Java grammar) of the wind control system, an external data source access request needing to be debugged is received, and the external data source access request is forwarded to a corresponding countermeasure stream interface; specifically, the debug interface may also provide for calling an engine module (a function of Java grammar) in the application, and in the decision flow, the debug interface may be called to debug external data that has not been verified yet;
s2): when a user initiates a transaction, calling is additionally carried out according to the configured scene set: in credit transaction, a user initiates a request at a client, sends a self-executed decision flow interface request to a front-end system for verification through a front-end system, and simultaneously calls a unified online data source debugging decision flow interface; specifically, the verification of the front-end system comprises identity person information and data validity;
s3): decision flow of call: exposing an online data source debugging decision flow interface, and receiving a transmitted request parameter into a decision flow, wherein an application module is an engine module, and the Controller module calls and receives the transmitted request parameter into the decision flow;
s4): according to different scene codes, branching to different decision streams: the method comprises the steps that a unified online data source debugging decision flow is configured with different branches according to different scene codes, a request parameter sent by a Controller is provided with bussiness codes (unified numbers), and the decision flow enters different business branch flows according to the corresponding bussiness codes and flows under the current branches; specifically, in the step S4, the bussiness code is divided into the following branches:
channel 1: trust; channel 2: letter use; channel 3: and (5) adjusting the forehead.
If businessCode is channel 1, then the request for debugging the external data source only goes to the branch where channel 1 is located and flows to the next node in branch 1;
s5): in each different scene branch, firstly obtaining the configuration of the number control of the sample to be debugged under the scene; specifically, after entering a branch flow corresponding to the businessCode, the first node is the configuration of the number of samples and the expiration time of the external number source needed to be debugged by the branch, and when the node is reached, the number of samples, the expiration time and the batch information needed to be configured at this time can be obtained;
such as apicode=ws_br_035, count=50000, expiretime=20230731, batch num=1, and then pass the data down.
S6): invoking an external data source through HTTP (a computer protocol) of the system itself, wherein in the process of invoking the data source, only the selected data source is invoked through the external data invoking module;
s7): the data source returns corresponding query results, the results are assigned to the decision flow, and rule configuration is carried out according to the need; specifically, when the external debugging data source calling node is executed, the intelligent decision engine calls a thirdData module (a function of Java grammar), and when the intelligent decision engine is configured into a plurality of data sources, the data sources with the most total time consumption are called through a thread pool in parallel; the destination address of the external number call is a unified DSM (digital surface model ), after the DSM receives the request of the decision engine, the DSM routes to different external number interfaces according to the apiCode, and then requests the external number manufacturer interface according to the transmitted client information, so as to obtain interface service information provided by the external number manufacturer in real time, and assigns the information to an external number unified format and returns the information, wherein the information also comprises sample number configuration information transmitted by a thirdData module;
s8): asynchronous database-falling query results and intermediate variable data through a message queue: after the related foreign number debugging information is queried from the DSM, the database interaction module adopted by the consumer side is a DBdata module (a function of Java grammar) through a message queue, a database is interacted with Oracle through a persistence component, and then the fields to be stored are stored in a structured data table of the Oracle. Specifically, the producer is a decision engine, and the message body delivered includes the queried debug external data source information and intermediate variables generated in the decision flow, wherein the table fields are defined by application components, and then the database table is created for structured storage.
The foregoing description is only illustrative of the present invention and is not intended to limit the scope of the invention, and all equivalent structural changes made by the present invention and the accompanying drawings, or direct or indirect application in other related technical fields, are included in the scope of the present invention.
Claims (9)
1. A multi-data source online analysis test method in credit scene is characterized in that: the method comprises the following steps:
s1): establishing a unified external data debugging interface: an interface capable of uniformly calling a fixed decision stream is newly added in a Controller module of the wind control system, an external data source access request needing to be debugged is received, and the external data source access request is forwarded to a corresponding countermeasure stream interface;
s2): when a user initiates a transaction, calling is additionally carried out according to the configured scene set: in credit transaction, a user initiates a request at a client, sends a self-executed decision flow interface request to a front-end system for verification through a front-end system, and simultaneously calls a unified online data source debugging decision flow interface;
s3): decision flow of call: exposing an online data source debugging decision flow interface, and receiving a transmitted request parameter into a decision flow, wherein an application module is an engine module, and the Controller module calls and receives the transmitted request parameter into the decision flow;
s4): according to different scene codes, branching to different decision streams: the unified online data source debugging decision flow is configured with different branches according to different scene codes, the request parameters sent by the Controller are provided with businesscodes, and the decision flow enters different business branch flows according to the corresponding businesscodes and flows under the current branches;
s5): in each different scene branch, serially acquiring configuration of the number control of the samples to be debugged in the scene through a system flow;
s6): invoking the required data source node at the decision flow: the external data source is called through the HTTP of the system, and in the flow of the data source debugging, the selected data source is called only through the external data calling module;
s7): the data source returns corresponding query results, the results are assigned to the decision flow, and rule configuration is carried out according to the need;
s8): asynchronous database-falling query results and intermediate variable data through a message queue: after the related foreign number debugging information is queried from the DSM, a database interaction module adopted by a consumer side is a DBdata module through a message queue, and after interaction with an Oracle through a persistence component mybatis, the fields to be stored are stored in a structured data table of the Oracle.
2. The method for online analysis and testing of multiple data sources in a credit scenario according to claim 1, wherein: in the step S1, the debug interface may also provide for calling an engine module in the application, and in the decision flow, the debug interface may be called to debug external data that has not been verified yet.
3. The method for online analysis and testing of multiple data sources in a credit scenario according to claim 1, wherein: in the step S2, the verification of the front-end system comprises identity information and data validity.
4. The method for online analysis and testing of multiple data sources in a credit scenario according to claim 1, wherein: in the step S4, the bussinesscode is divided into the following branches:
channel 1: trust;
channel 2: letter use;
channel 3: and (5) adjusting the forehead.
5. The method for online analysis and testing of multiple data sources in a credit scenario according to claim 1, wherein: in the step S5, after entering a branch flow corresponding to the businessCode, the first node is the configuration of the number of samples and the expiration time of the external number source required to be debugged by the branch, and when the node is walked to, the number of samples and the expiration time and the batch information required to be configured at this time can be obtained.
6. The method for online analysis and testing of multiple data sources in a credit scenario according to claim 1, wherein: in the step S7, when the external debugging data source call node is executed, the intelligent decision engine calls the thirdData module, and when the intelligent decision engine is configured into a plurality of data sources, the data source with the most time consumption is called through the thread pool in parallel.
7. The method for online analysis and testing of multiple data sources in a credit scenario according to claim 6, wherein: the destination address of the external number call is a unified DSM, the DSM receives the request of the decision engine, and then routes the request to different external number interfaces according to the apiCode, and then requests the external number manufacturer interface according to the transmitted client information, so as to obtain interface service information provided by the external number manufacturer in real time, assign the information to an external number unified format and return the information, wherein the information also comprises sample number configuration information transmitted by the thirdData module.
8. The method for online analysis and testing of multiple data sources in a credit scenario according to claim 1, wherein: in the step S8, the producer is a decision engine, and the transmitted message body includes the queried debug external data source information and the intermediate variables generated in the decision stream.
9. The method for online analysis and testing of multiple data sources in a credit scenario according to claim 1, wherein: in the step S8, a component is applied, a table field is defined, and then a database table is created for structured storage.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410017928.4A CN117785982A (en) | 2024-01-05 | 2024-01-05 | Multi-data source online analysis test method under credit scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410017928.4A CN117785982A (en) | 2024-01-05 | 2024-01-05 | Multi-data source online analysis test method under credit scene |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117785982A true CN117785982A (en) | 2024-03-29 |
Family
ID=90381613
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410017928.4A Pending CN117785982A (en) | 2024-01-05 | 2024-01-05 | Multi-data source online analysis test method under credit scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117785982A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118134616A (en) * | 2024-05-08 | 2024-06-04 | 蓝象智联(杭州)科技有限公司 | Combined rule product design method and system based on privacy protection |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB201419630D0 (en) * | 2013-11-04 | 2014-12-17 | Global Analytics Inc | Real-time adaptive decision system and method using predictive modeling |
| CN111638948A (en) * | 2020-06-03 | 2020-09-08 | 重庆银行股份有限公司 | Multi-channel high-availability big data real-time decision making system and decision making method |
| CN115687485A (en) * | 2022-11-11 | 2023-02-03 | 奇瑞徽银汽车金融股份有限公司 | Multi-data-source management system and method applied to automobile finance |
-
2024
- 2024-01-05 CN CN202410017928.4A patent/CN117785982A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB201419630D0 (en) * | 2013-11-04 | 2014-12-17 | Global Analytics Inc | Real-time adaptive decision system and method using predictive modeling |
| CN111638948A (en) * | 2020-06-03 | 2020-09-08 | 重庆银行股份有限公司 | Multi-channel high-availability big data real-time decision making system and decision making method |
| CN115687485A (en) * | 2022-11-11 | 2023-02-03 | 奇瑞徽银汽车金融股份有限公司 | Multi-data-source management system and method applied to automobile finance |
Non-Patent Citations (2)
| Title |
|---|
| RAONI LOURENÇO等: "BugDoc: A System for Debugging Computational Pipelines", SIGMOD \'20: PROCEEDINGS OF THE 2020 ACM SIGMOD INTERNATIONAL CONFERENCE ON MANAGEMENT OF DATA, 31 March 2020 (2020-03-31), pages 2733 - 2736 * |
| 陈稀: "基于深度学习的智能风控系统", 中国优秀硕士学位论文全文数据库 (信息科技辑), 15 January 2019 (2019-01-15), pages 140 - 186 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118134616A (en) * | 2024-05-08 | 2024-06-04 | 蓝象智联(杭州)科技有限公司 | Combined rule product design method and system based on privacy protection |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN117785982A (en) | Multi-data source online analysis test method under credit scene | |
| CN110856038B (en) | Video generation method and system, and storage medium | |
| CN106020948B (en) | A kind of process dispatch method and device | |
| CN107645562A (en) | Data transmission processing method, device, equipment and system | |
| CN105426292A (en) | Game log real-time processing system and method | |
| WO2022105153A1 (en) | Risk control decision-making method and apparatus, device, and storage medium | |
| CN101377759A (en) | Automatic interface test system | |
| CN110321273A (en) | A kind of business statistical method and device | |
| CN111028007B (en) | User portrait information prompting method, device and system | |
| CN112200465B (en) | Electric power AI method and system based on multimedia information intelligent analysis | |
| CN110515827A (en) | Automated testing method, device, computer equipment and storage medium | |
| CN106325988A (en) | Task scheduling method and device | |
| CN109460307A (en) | Micro services a little, which are buried, based on log calls tracking and its system | |
| CN112527421A (en) | Service calling method, device, equipment and storage medium | |
| CN112785124A (en) | Method and system for auditing compliance of telecommunication service | |
| CN106452815A (en) | Informatization management method, device and system | |
| CN113946363A (en) | Method and device for executing and configuring service data, computer equipment and storage medium | |
| CN111161052B (en) | Bank operation data processing method and device | |
| CN104216986B (en) | The device and method of pre-operation raising efficiency data query is carried out with the data update cycle | |
| CN114371884A (en) | Method, device, equipment and storage medium for processing Flink calculation task | |
| CN115168297A (en) | Bypassing log auditing method and device | |
| CN112468340A (en) | Pre-audit business data configuration system for multiple tenants | |
| CN114900569B (en) | Test script acquisition method and device | |
| CN113709314B (en) | Intelligent seat outbound method and device, electronic equipment and computer storage medium | |
| CN117216062B (en) | Information management system based on data coding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |