CN115827050A - Data calling method and device, electronic equipment and storage medium - Google Patents

Abstract

The application provides a data calling method, a data calling device, electronic equipment and a storage medium, wherein the method comprises the following steps: receiving a calling request of a calling party, wherein the calling request comprises calling configuration information; the calling configuration information is configured in advance by a calling party and comprises calling party information and interface configuration information corresponding to the calling party information; generating a calling task according to the calling request; and analyzing the interface configuration information corresponding to the calling party information according to the calling party information in the calling task, and responding to the calling request. The method comprises the steps that a calling party configures calling configuration information in advance, a calling request is initiated according to the calling configuration information, and a calling task is generated after the calling request is received; and responding to the calling request according to the calling party information and the corresponding interface configuration information. The condition of redesigning the service code is not needed, the condition of error test due to code change is reduced, and the efficiency of data calling is improved.

Description

Data calling method and device, electronic equipment and storage medium

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a data calling method, an apparatus, an electronic device, and a storage medium.

Background

With the rapid development of enterprise digital transformation, enterprises need more and more data for various services, and because the interfaces to which the enterprises need to be connected and the data processing and processing indexes are also very different, different interface accesses and data index processing of the enterprises face huge challenges. At present, when an interface to be connected is changed due to business change of an enterprise, a business code in a system is usually redesigned and developed, and is re-released to be on-line after being tested, so that the mode can easily cause testing errors, and the efficiency is low.

Disclosure of Invention

The embodiment of the invention aims to provide a data calling method, a data calling device, electronic equipment and a storage medium.

In a first aspect, an embodiment of the present application provides a data calling method, including: receiving a calling request of a calling party, wherein the calling request comprises calling configuration information; the calling configuration information is configured in advance by a calling party and comprises calling party information and interface configuration information corresponding to the calling party information; generating a calling task according to the calling request; and analyzing the interface configuration information corresponding to the calling party information according to the calling party information in the calling task, and responding to the calling request.

In the implementation process, a calling party configures calling configuration information in advance, initiates a calling request according to the calling configuration information, and generates a calling task after receiving the calling request; and responding to the calling request according to the calling party information and the corresponding interface configuration information. The condition of redesigning the service code is not needed, the condition of error test caused by code change is reduced, and the efficiency of data calling is improved.

Optionally, in this embodiment of the present application, the interface configuration information includes an interface call flow and a data processing script; analyzing interface configuration information corresponding to the caller information according to the caller information in the calling task, and responding to the calling request, wherein the method comprises the following steps: according to the calling party information in the calling task, inquiring an interface calling flow corresponding to the calling party information and a data processing script corresponding to each interface; and analyzing the data processing script corresponding to each interface according to the interface calling flow so as to execute a calling task and respond to the calling request.

In the implementation process, the corresponding interface calling flow and the data processing script corresponding to each interface are determined by calling the calling party information in the task, and after the rule is analyzed, the task is processed and the calling request is responded. The user can complete the data calling and the data calling efficiency by configuring the interface information.

Optionally, in this embodiment of the present application, after generating the invocation task according to the invocation request, the method further includes: adding the calling task into a priority queue; carrying out pre-judgment processing on the calling tasks in the priority queue, and confirming whether the calling tasks meet execution conditions; if so, according to the priority queue, analyzing interface configuration information corresponding to the calling party information according to the calling party information in the calling task.

In the implementation process, after a task request of a calling party comes in, the task request is preferentially put into a priority queue, and if the calling task does not have any priority attribute, consumption execution can be carried out according to the processing sequence of the queue; when a business scene of a certain organization temporarily needs a quick response business, and even if the business scene is arranged at the last, the task needs to be executed preferentially, the priority queue can help to complete the requirement, and the quick response request is realized.

Optionally, in this embodiment of the present application, adding the call task into the priority queue includes: if the data source corresponding to the calling configuration information is in a service stop state, setting the task state of the calling task of the calling party as a suspension state; and if the data source is not in the service halt state, adding the calling task of which the task state is the suspension into the priority queue.

In the implementation process, if the data source corresponding to the calling configuration information is in the service halt state, the task state of the calling task of the calling party is set to be suspended, so that a large amount of overtime and failure are avoided, after the data source is recovered, the task state of the calling task is timely adjusted and added into the priority queue for execution, and the usability of the data processing platform is improved.

Optionally, in this embodiment of the present application, the pre-judgment processing includes power equal judgment processing and current limit judgment processing; the method for pre-judging the calling tasks in the priority queue and confirming whether the calling tasks meet the execution conditions comprises the following steps: generating a request serial number according to the calling request; performing idempotent judgment processing on the calling request according to the request serial number to obtain an idempotent judgment result; judging whether the calling request meets a flow execution condition or not according to preset flow configuration data of each calling party, and obtaining a flow judgment result; and confirming whether the calling task meets the execution condition or not according to the idempotent judgment result and the flow judgment result.

In the implementation process, before the calling task is executed, the power and the like judgment processing and the flow judgment processing are firstly carried out on the task, the current limiting and the suspension processing are carried out on the calling of different mechanism tasks, different current limiting processing is adopted for different calling parties, the effective management of the complex task processing and the interface calling is realized, the performance of a data processing platform is optimized, and the efficiency of data calling is improved.

Optionally, in this embodiment of the present application, the flow determination result includes a flow excess result and a flow qualified result; judging whether the calling request meets a flow execution condition according to preset flow configuration data of each calling party, and obtaining a flow judgment result, wherein the flow judgment result comprises the following steps: obtaining the number of calling requests of a calling party and the priority of the calling requests; configuring flow configuration data of a calling party through an isolation algorithm; judging whether the quantity of the calling requests of the calling party exceeds the corresponding flow configuration data, if so, representing that the calling requests do not meet flow execution conditions, adjusting the priority of the calling requests to be low, and generating a flow excess result; if not, the representation calling request meets the flow execution condition, and a flow qualified result is generated.

In the implementation process, a certain isolation algorithm is adopted for the situation that the flow of a single calling party is large or the situation that the task of the calling party is processed for a long time, so that all calling parties are guaranteed to have relative resources to respond, the situation that the sufficient resources are allocated to the calling parties with larger requests can be guaranteed, and the normal services of other calling parties cannot be influenced when the traffic of one calling party is suddenly increased.

Optionally, in this embodiment of the present application, the call configuration information includes a callback address and timeout time; after analyzing the interface configuration information corresponding to the calling party information according to the calling party information in the calling task and responding to the calling request, the method further comprises the following steps: obtaining a query response result corresponding to the calling task; and sending the query response result to the calling party through an asynchronous callback mechanism according to the callback address and the timeout time.

In the implementation process, the query response result is sent to the caller through an asynchronous callback mechanism, so that the problems that a large number of interfaces need to be called in a task and time consumption is long, IO (input/output) blocks the task due to the adoption of a traditional synchronous request response mode, and resources of the caller and the callee are wasted are solved.

In a second aspect, an embodiment of the present application further provides a data invoking device, including: the receiving request module is used for receiving a calling request, and the calling request comprises calling configuration information; the calling configuration information comprises calling party information and interface configuration information corresponding to the calling party information; the task generation module is used for generating a calling task according to the calling request; and the response module is used for analyzing the interface configuration information corresponding to the calling party information according to the calling party information in the calling task and responding to the calling request.

In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor and a memory, the memory storing processor-executable machine-readable instructions, the machine-readable instructions when executed by the processor performing the method as described above.

In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor, and the computer program performs the method described above.

By adopting the data calling method, the data calling device, the electronic equipment and the storage medium, the calling party configures calling configuration information in advance, initiates a calling request according to the calling configuration information, and generates a calling task after receiving the calling request; and responding to the calling request according to the calling party information and the corresponding interface configuration information. The condition of redesigning the service code is not needed, the condition of error test due to code change is reduced, and the efficiency of data calling is improved. The task state of the calling task of the calling party is set to be suspended, so that a large amount of overtime and failure are avoided, a certain isolation algorithm is adopted, all calling parties are guaranteed to have relative resources to respond, sufficient resources are guaranteed to be allocated to the calling parties with larger requests, and normal services of other calling parties cannot be influenced when the traffic of one calling party is suddenly increased.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.

Fig. 1 is a schematic flowchart of a data call method according to an embodiment of the present application;

fig. 2 is a schematic structural diagram of call configuration information provided in an embodiment of the present application;

fig. 3 is a schematic diagram of a task execution flow of a data processing platform according to an embodiment of the present application

Fig. 4 is a schematic structural diagram of a data call device according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following examples are merely used to more clearly illustrate the technical solutions of the present application, and therefore are only examples, and the protection scope of the present application is not limited thereby.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

In the description of the embodiments of the present application, the technical terms "first", "second", and the like are used only for distinguishing different objects, and are not to be construed as indicating or implying relative importance or implicitly indicating the number, specific order, or primary-secondary relationship of the technical features indicated. In the description of the embodiments of the present application, "a plurality" means two or more unless specifically defined otherwise.

Please refer to fig. 1, which is a schematic flow chart diagram of a data calling method according to an embodiment of the present application. The data calling method provided by the embodiment of the application can be applied to a data processing platform, and the data processing platform can comprise a terminal and a server; the terminal may be a smart phone, a tablet computer, a Personal Digital Assistant (PDA), or the like; the server may specifically be an application server, and may also be a Web server.

Step S110: receiving a calling request of a calling party, wherein the calling request comprises calling configuration information; the calling configuration information is configured by a calling party and comprises calling party information and interface configuration information corresponding to the calling party information.

The embodiment of the step S110 includes: the data processing platform receives a calling request of a calling party, wherein the calling party can be a service or a mechanism needing to call data through a data interface, and the calling request can be initiated to the data processing platform, for example, a newly online mechanism needs to call data of a three-party platform for support; or the on-line mechanism has service change, and the corresponding interface needs to be changed, and a calling request can be sent to the data processing platform. The communication mode of the calling party and the data processing platform can adopt http protocol or RabbitMq communication.

The calling request comprises calling configuration information; the calling configuration information is pre-configured by the caller. Before the calling direction data processing platform sends a calling request, calling configuration information can be configured in advance according to service requirements, and the calling configuration information comprises calling party information and interface configuration information corresponding to the calling party information. The caller information may include the name of the caller, the calling mode, the product identification of the caller service, etc.; the interface configuration information includes attributes of an interface to be called, a calling interface flow and the like.

Step S120: and generating a calling task according to the calling request.

The embodiment of the step S120 includes: and after receiving the calling request sent by the calling rice, the data processing platform generates a calling task corresponding to the calling request and stores the related information of the calling task.

Step S130: and analyzing the interface configuration information corresponding to the calling party information according to the calling party information in the calling task, and responding to the calling request.

The embodiment of the step S130 includes: acquiring interface configuration information corresponding to the calling party information according to the calling party information in the calling task, and analyzing the interface configuration information corresponding to the user information, wherein the analyzing process comprises the steps of analyzing, reading and configuring, analyzing rules in configuration and the like, the calling task is completed, and a response result is generated; and storing the response result and returning the response result to the caller.

In the implementation process, a calling party configures calling configuration information in advance, initiates a calling request according to the calling configuration information, and generates a calling task after receiving the calling request; and responding to the calling request according to the calling party information and the corresponding interface configuration information. The condition of redesigning the service code is not needed, the condition of error test due to code change is reduced, and the efficiency of data calling is improved.

Please refer to fig. 2, which illustrates a schematic structural diagram of the calling configuration information provided in the embodiment of the present application.

In an optional embodiment, before sending the call request to the data processing platform, the caller first performs configuration of call configuration information on the data processing platform, where the call configuration information includes caller information and interface configuration information corresponding to the caller information. The configuration can be divided into several steps, and each step can configure the information of one interface or the information of a plurality of interfaces. The steps are executed in series according to the sequence, if a plurality of interfaces exist in one step, the steps are executed concurrently, and parameter transmission can be completed through the context.

Specifically, for example, as shown in FIG. 2, visual interface configuration information is provided at a data processing platform. The visual interface can be realized by Vue, and Vue is a set of progressive framework for constructing the user interface. The caller information includes organization and product numbers and business names. And determining corresponding unique interface configuration information according to the organization number, the product number and the service name.

The interface configuration information comprises interface basic attributes, data index mapping field rules and data processing script configuration. The interface basic attribute comprises parameters such as interface name, interface path, interface description, interface data caching time, timeout time, suspension state code, retry times, caching time and the like. The data pointer mapping field rule includes a field name, a data mapping path, a data field type, and a field description. The data processing script configuration comprises a request data processing script of the interface request required data, a response data processing script of the interface response data, a global context and a script automation test tool.

In the implementation process, interface calling flow rules corresponding to each calling party and uploaded data processing execution scripts are configured according to dimensions such as different mechanism numbers, product numbers, flow names and the like, meanwhile, an automatic script testing tool is provided, interface calling flows and data index processing logics of different mechanisms can be uniformly and efficiently managed, core codes and logics of different mechanisms for interface calling and data index processing are isolated, frequent updating and core code adding are avoided, frequent edition sending is avoided, a script automatic testing tool is used for replacing a traditional testing method, and the efficiency of new mechanism access and stock business demand change is improved.

Optionally, in this embodiment of the present application, the interface configuration information includes an interface call flow and a data processing script; analyzing interface configuration information corresponding to the caller information according to the caller information in the calling task, and responding to the calling request, wherein the method comprises the following steps: according to the calling party information in the calling task, inquiring an interface calling flow corresponding to the calling party information and a data processing script corresponding to each interface; and analyzing the data processing script corresponding to each interface according to the interface calling flow so as to execute a calling task and respond to the calling request.

In the specific implementation process: the interface configuration information comprises an interface calling flow and a data processing script; the data processing platform provides tools on a page to a user to configure a related interface, and the data processing platform comprises a plurality of interface calling steps, and the interface responds to the uploading of a data processing logic script. According to the calling party information in the calling task, an interface calling flow corresponding to the calling party information and a data processing script corresponding to each interface are inquired, rules are analyzed through a rule analysis engine according to the interface calling flow, the data processing script corresponding to each interface is analyzed, a script execution engine executes the script, and data pulling, data index processing, data type verification and data path mapping are performed respectively in a concurrent or serial mode.

Specifically, for example, the configuration is analyzed, the request parameter processing script is executed according to different rules to pull data from a corresponding data source, the data processing script is executed to obtain a data index verification result, any processing result is integrated, and the result is returned to the caller.

In the implementation process, the corresponding interface calling flow and the data processing script corresponding to each interface are determined by calling the calling party information in the task, and after the rule is analyzed, the task is processed and the calling request is responded. The user can realize the calling of the data and the efficiency of the data calling only by configuring the interface information.

Optionally, in this embodiment of the application, after the invoking task is generated according to the invoking request, the method further includes: adding the calling task into a priority queue; carrying out pre-judgment processing on the calling tasks in the priority queue, and confirming whether the calling tasks meet execution conditions; if so, according to the priority queue, analyzing interface configuration information corresponding to the calling party information according to the calling party information in the calling task.

In the specific implementation process: according to the processing platform, after receiving the task call, the relevant information of the task is stored, then the task is distributed to a priority queue, and the priority level of the call task is obtained. The priority level of the task can be preset by the caller, and can also be judged according to the transfer information configured by the caller. Specifically, for example, for a caller task that needs to respond quickly, the priority level of the corresponding caller task may be adjusted to a high priority, and the data processing platform preferentially consumes the caller task with the high priority.

The pre-judgment processing is carried out on the calling tasks in the priority queue, whether the calling tasks meet the execution condition is confirmed, and the pre-judgment processing can comprise power judgment, whether the mechanism has limited current, whether the mechanism needs to be hung and the like. And if the calling task meets the execution condition after the judgment and the processing, executing the step of analyzing the interface configuration information corresponding to the calling party information according to the calling party information in the calling task according to the priority level of the scheduling task in the priority queue.

In the implementation process, after a task request of a calling party comes in, the task request is preferentially put into a priority queue, and if the calling task does not have any priority attribute, consumption execution can be carried out according to the processing sequence of the queue; when a business scene of an organization temporarily needs a quick response business and even if the business scene is arranged at the last, the task of the business scene needs to be executed preferentially, the priority queue can help to fulfill the requirement and realize quick response requests.

Optionally, in this embodiment of the present application, adding the call task into the priority queue includes: if the data source corresponding to the calling configuration information is in a service stop state, setting the task state of the calling task of the calling party as a suspension state; and if the data source is not in the service halt state, adding the calling task of which the task state is suspended into the priority queue.

In the specific implementation process: and if the data source corresponding to the calling configuration information is in the service stop state, setting the task state of the calling task of the calling party to be suspended. In some scenarios, if a data source of a three-party interface docked by a caller is in a service suspension update stage, data of the data source cannot be called through the data interface, but at this time, a call task of the caller may still initiate a request, and the three-party interface is called to obtain related data for calculation, which may cause a lot of timeout and failures. Therefore, the task state of all or part of the calling tasks of the calling party can be set to be suspended according to the calling party information of the calling party, such as the mechanism number, and after the setting is successful, all the task requests of the calling party cannot continue to execute the corresponding calling requests after coming in, but the tasks are stored in the database.

And after the data source is confirmed not to be in the service stop state, adding the calling task in the task state of suspension into the priority queue, and after the three-party interface is updated and the service is restarted, the data processing platform releases the suspension of the task of the calling party, and all the suspended tasks can be queued again for execution.

In the implementation process, if the data source corresponding to the calling configuration information is in the service halt state, the task state of the calling task of the calling party is set to be suspended, so that a large amount of overtime and failure are avoided, after the data source is recovered, the task state of the calling task is timely adjusted and added into the priority queue for execution, and the usability of the data processing platform is improved.

Optionally, in this embodiment of the present application, the pre-judgment processing includes idempotent judgment processing and current-limiting judgment processing; the method comprises the following steps of carrying out pre-judgment processing on a calling task in a priority queue, and confirming whether the calling task meets an execution condition or not, wherein the pre-judgment processing comprises the following steps: generating a request serial number according to the call request; performing idempotent judgment processing on the calling request according to the request sequence number to obtain an idempotent judgment result; judging whether the calling request meets a flow execution condition or not according to preset flow configuration data of each calling party to obtain a flow judgment result; and confirming whether the calling task meets the execution condition or not according to the idempotent judgment result and the flow judgment result.

In the specific implementation process: the pre-judgment processing comprises power equal judgment processing and current limiting judgment processing. Idempotency is a commitment of system interfaces that multiple identical inputs will have the same result feedback and equivalent processing impact as long as the call to the interface is successful. In order to prevent a calling party from initiating a calling request for many times, the request submitted by rereading is filtered out through idempotent judgment processing.

Specifically, for example, according to the call request, a request sequence number is generated based on the request parameter; and performing idempotent judgment processing on the calling request according to the request sequence number to generate an idempotent judgment result, wherein the idempotent judgment result comprises an idempotent success judgment result and an idempotent failure judgment result. In a preset time period, the same interface receives a call request with the same request serial number, the call request is considered to be a repeated request, an idempotent judgment failure result is generated, otherwise, an idempotent judgment success result is generated, and the repeated request cannot be continuously executed.

And judging whether the calling request meets a flow execution condition or not according to preset flow configuration data of each calling party, and obtaining a flow judgment result. The preset flow configuration data of each caller comprises a fixed number of requests that can be admitted by each caller within a fixed time window. If the request quantity of a calling party exceeds the corresponding flow configuration data in a fixed time window, the task is not continuously executed, but the calling task of the calling party is added into the queue again, and a result representing the failure of flow judgment is generated.

And confirming whether the calling task meets the execution condition or not according to the idempotent judgment result and the flow judgment result. For example, if the idempotent judgment result and the flow judgment result are successfully characterized or qualified, the task is called to meet the execution condition, and the execution steps can be continued.

In the implementation process, before the calling task is executed, the tasks are subjected to power and other judgment processing and flow judgment processing, the tasks of different mechanisms are called to be subjected to current limiting, suspension and other processing, different current limiting processing is adopted for different calling parties, the complex task processing and interface calling are effectively managed, the performance of a data processing platform is optimized, and the data calling efficiency is improved.

Optionally, in this embodiment of the present application, the flow determination result includes a flow excess result and a flow qualified result; judging whether the calling request meets a flow execution condition according to preset flow configuration data of each calling party to obtain a flow judgment result, wherein the flow judgment result comprises the following steps: obtaining the number of calling requests of a calling party and the priority of the calling requests; configuring flow configuration data of a calling party through an isolation algorithm; judging whether the quantity of the calling requests of the calling party exceeds the corresponding flow configuration data, if so, representing that the calling requests do not meet flow execution conditions, adjusting the priority of the calling requests to be low, and generating a flow excess result; if not, the representation calling request meets the flow execution condition, and a flow qualified result is generated.

In the specific implementation process: obtaining the number of the calling requests of a calling party and the priority of the calling requests; configuring flow configuration data of a calling party through an isolation algorithm; specifically, for example, a fixed number of requests may be allowed to enter within a time window set for each caller, respectively. And if the request quantity of the calling party exceeds the flow configuration data corresponding to the calling party in the fixed time window, rejecting the task or rejoining the task into the queue, and the time window continuously slides backwards along with the lapse of time. When the number of the calling requests is set for the calling party, the allowed request quantity in a unit time window can be configured according to a service scene, if the request quantity of the calling party exceeds the flow configuration data corresponding to the calling party, the representation calling request does not meet the flow execution condition, the priority of the calling request is adjusted to be low, and a flow excess result is generated; if not, the representation calling request meets the flow execution condition, and a flow qualified result is generated.

In the implementation process, a certain isolation algorithm is adopted for the situation that the flow of a single calling party is large or the situation that the task of the calling party is processed for a long time, so that all calling parties are guaranteed to have relative resources to respond, the situation that the sufficient resources are allocated to the calling parties with larger requests can be guaranteed, and the normal services of other calling parties cannot be influenced when the traffic of one calling party is suddenly increased.

Optionally, in this embodiment of the present application, the call configuration information includes a callback address and timeout time; after analyzing the interface configuration information corresponding to the calling party information according to the calling party information in the calling task and responding to the calling request, the method further comprises the following steps: obtaining a query response result corresponding to the calling task; and sending the query response result to the calling party through an asynchronous callback mechanism according to the callback address and the timeout time.

In the specific implementation process: and acquiring a query response result corresponding to the calling task, wherein the query response result comprises a processing result of the calling task, and the query response result is sent to the calling party through an asynchronous callback mechanism according to the callback address and the timeout time. For example, if a task sent by a caller is executed, the data gateway notifies the caller of an execution result according to a callback address in the task; the caller can actively inquire the result after the service sends the message; if the caller adds a timeout to the request, the query response result needs to be returned to the caller within a specified time, regardless of the query result.

An asynchronous callback mechanism is adopted when data are returned to a calling party, the calling party does not need to be blocked after sending a task request, the calling party can be informed to the calling party actively after the data processing platform processes the task, and meanwhile, if the data processing platform does not inform the result for a long time, the calling party can also obtain the current task state through an actively-queried interface.

In the implementation process, the query response result is sent to the calling party through an asynchronous callback mechanism, so that the condition that a large number of interfaces need to be called in a task and time consumption is long, IO (input/output) is blocked by adopting a traditional synchronous request response mode, and resources of the two parties are wasted is improved.

In an optional embodiment, after analyzing the interface configuration information corresponding to the caller information according to the caller information in the call task and responding to the call request, the method further includes: acquiring interface processing data of each interface of a calling party through data embedding points; and processing data according to the interface of each interface to generate a monitoring index data chart.

In a specific implementation process, in a process of executing service processing by a data processing platform, before and after calling a request interface, the passing rate, the response rate and the blocking rate of each interface of each calling party are respectively counted in a point burying mode, and a monitoring index data chart is generated. The monitoring index data graph can be a real-time graph and is generated by integrating a promethus framework, and the promethus is an open-source system monitoring and alarming system. The calling party can observe the response state and the speed of the business interface in real time through the chart, adjust the interface configuration and the like, and feed back the interface response condition to the enterprise providing the three-party data interface. The statistical processing of relevant calling indexes, the storage of the interface response original text and the data processing result and the regular big data filing are realized.

Please refer to fig. 3 for a schematic diagram of a task execution flow of a data processing platform according to an embodiment of the present application.

In an optional embodiment, the caller pre-configures the call configuration information in the data gateway, the interface and the script configuration rule are stored by mongodb, and the script execution engine adopts jython-standby. And initiating a calling request to the data processing platform according to the calling configuration information. And after receiving the call request, the data processing platform generates a call task and adds the call task into the value priority queue. And if the flow of the calling party reaches the upper flow limit, namely the execution condition is not met, the calling task is added into the queue again. If the execution condition is met, continuing to judge that the calling party corresponding to the execution condition is suspended, and if the calling party is in a suspended state, temporarily storing the task to the database MySql; and if the calling party releases the suspended state, the calling tasks meeting the execution conditions in the database are added into the queue again. In the task execution, a queue stored by an abnormal time and a service suspension delay 9s is adopted as RabbitMq, wherein the delay time can be set according to the actual situation. The abnormal and suspended queue consumers adopt springboot based RabbitMq configured consumers. The merchant current-limiting suspension configuration storage medium adopts zookeeper, the medium for storing the data processing and processing index according to certain cache time after the task execution is finished is Redis, and the MySql is adopted for storing the historical execution task data.

And if the calling party is not suspended, continuing to execute the task, reading calling configuration information corresponding to the task, analyzing the rule through a rule analysis engine, and respectively performing data pulling, data index processing, data type checking and data path mapping in a concurrent or serial mode. And then assembling the request to obtain the access data, the anti-fraud and the wind control query result. And storing the query result, wherein the query result can be stored in a database.

In the process of executing business processing by a data processing platform, the passing rate, the response rate and the blocking rate of each interface of each mechanism are counted in a data buried point counting mode, and monitoring index data are generated through an integrated proxy framework to form a real-time chart for displaying. The calling party can observe the response state and the speed of the business interface in real time through the chart, adjust the interface configuration and the like, and feed back the interface response condition to the enterprise providing the three-party data interface. And carrying out statistics on call related indexes, storing interface response original texts and data processing results, and filing large data regularly. The data gateway informs the caller of executing the result according to the callback address in the task, and the caller can actively inquire the result according to a certain strategy after the service sends the message.

Please refer to fig. 5, which is a schematic structural diagram of a data call device according to an embodiment of the present application; the embodiment of the present application provides a data calling apparatus 200, including:

a receiving request module 210, configured to receive a call request, where the call request includes call configuration information; the calling configuration information comprises calling party information and interface configuration information corresponding to the calling party information;

a task generation module 220, configured to generate a call task according to the call request;

and the response module 230 is configured to analyze the interface configuration information corresponding to the caller information according to the caller information in the call task, and respond to the call request.

Optionally, in this embodiment of the present application, the data calling device, the interface configuration information includes an interface calling process and a data processing script; the response module 230 is specifically configured to query, according to the caller information in the call task, an interface call flow corresponding to the caller information and a data processing script corresponding to each interface; and analyzing the data processing script corresponding to each interface according to the interface calling flow so as to execute a calling task and respond to the calling request.

Optionally, in this embodiment of the application, the data invoking device further includes: the condition judgment module is used for adding the calling task into the priority queue; carrying out pre-judgment processing on the calling tasks in the priority queue, and confirming whether the calling tasks meet execution conditions; if so, according to the priority queue, analyzing interface configuration information corresponding to the calling party information according to the calling party information in the calling task.

Optionally, in this embodiment of the application, the data calling device and the condition determining module are further configured to set a task state of a calling task of a calling party to be suspended if a data source corresponding to the calling configuration information is in a service suspended state; and if the data source is not in the service halt state, adding the calling task of which the task state is suspended into the priority queue.

Optionally, in this embodiment of the application, the data invoking device, the pre-determination processing, includes an idempotent determination processing and a current-limiting determination processing; the condition judgment module is also used for generating a request serial number according to the call request; performing idempotent judgment processing on the calling request according to the request sequence number to obtain an idempotent judgment result; judging whether the calling request meets a flow execution condition or not according to preset flow configuration data of each calling party, and obtaining a flow judgment result; and confirming whether the calling task meets the execution condition or not according to the idempotent judgment result and the flow judgment result.

Optionally, in an embodiment of the present application, the data call device, and the flow determination result includes a flow excess result and a flow qualified result; the condition judgment module is also used for obtaining the number of the calling requests of the calling party and the priority of the calling requests; configuring flow configuration data of a calling party through an isolation algorithm; judging whether the quantity of the call requests of the calling party exceeds the corresponding flow configuration data, if so, representing that the call requests do not meet flow execution conditions, adjusting the priority of the call requests to be low, and generating a flow excess result; if not, the representation calling request meets the flow execution condition, and a flow qualified result is generated.

Optionally, in this embodiment of the present application, the data invoking device invokes the configuration information, where the configuration information includes a callback address and timeout time; the callback module is used for obtaining a query response result corresponding to the calling task; and sending the query response result to the calling party through an asynchronous callback mechanism according to the callback address and the timeout time.

It should be understood that the apparatus corresponds to the above-mentioned data calling method embodiment, and can perform the steps related to the above-mentioned method embodiment, and the specific functions of the apparatus can be referred to the above description, and the detailed description is appropriately omitted here to avoid redundancy. The device includes at least one software function that can be stored in memory in the form of software or firmware (firmware) or solidified in the Operating System (OS) of the device.

Please refer to fig. 5, which illustrates a schematic structural diagram of an electronic device according to an embodiment of the present application. An embodiment of the present application provides an electronic device 300, including: a processor 310 and a memory 320, the memory 320 storing machine readable instructions executable by the processor 310, the machine readable instructions when executed by the processor 310 performing the method as above.

The embodiment of the application also provides a storage medium, wherein the storage medium is stored with a computer program, and the computer program is executed by a processor to execute the method.

The storage medium may be implemented by any type of volatile or nonvolatile storage device or combination thereof, such as a Static Random Access Memory (SRAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), an Erasable Programmable Read-Only Memory (EPROM), a Programmable Read-Only Memory (PROM), a Read-Only Memory (ROM), a magnetic Memory, a flash Memory, a magnetic disk, or an optical disk.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The apparatus embodiments described above are merely illustrative and, for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

The above description is only an alternative embodiment of the embodiments of the present application, but the scope of the embodiments of the present application is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the embodiments of the present application, and all the changes or substitutions should be covered by the scope of the embodiments of the present application.

Claims (10)

1. A data calling method, comprising:

receiving a calling request of a calling party, wherein the calling request comprises calling configuration information; the calling configuration information is configured in advance by the calling party, and comprises calling party information and interface configuration information corresponding to the calling party information;

generating a calling task according to the calling request;

and analyzing interface configuration information corresponding to the calling party information according to the calling party information in the calling task, and responding to the calling request.

2. The method of claim 1, wherein the interface configuration information includes an interface call flow and a data processing script; analyzing interface configuration information corresponding to the caller information according to the caller information in the calling task, and responding to the calling request, wherein the method comprises the following steps:

according to the calling party information in the calling task, inquiring an interface calling flow corresponding to the calling party information and a data processing script corresponding to each interface;

and analyzing the data processing script corresponding to each interface according to the interface calling flow so as to execute the calling task and respond to the calling request.

3. The method of claim 1, wherein after generating a calling task from the call request, the method further comprises:

adding the calling task into a priority queue;

carrying out pre-judgment processing on the calling tasks in the priority queue, and confirming whether the calling tasks meet execution conditions; if yes, according to the priority queue, analyzing interface configuration information corresponding to the calling party information according to the calling party information in the calling task.

4. The method of claim 3, wherein adding the invocation task into a priority queue comprises:

if the data source corresponding to the calling configuration information is in a service stop state, setting the task state of the calling task of the calling party to be suspended;

and if the data source is not in the service halt state, adding the calling task of which the task state is the suspension into the priority queue.

5. The method of claim 3, wherein the pre-determination process comprises an idempotent determination process and a current limit determination process; carrying out pre-judgment processing on the calling tasks in the priority queue, and confirming whether the calling tasks meet execution conditions or not, wherein the pre-judgment processing comprises the following steps:

generating a request serial number according to the calling request;

performing power equal judgment processing on the calling request according to the request serial number to obtain a power equal judgment result;

judging whether the calling request meets a flow execution condition or not according to preset flow configuration data of each calling party, and obtaining a flow judgment result;

and confirming whether the calling task meets the execution condition or not according to the idempotent judgment result and the flow judgment result.

6. The method of claim 5, wherein the flow determination result comprises a flow excess result and a flow qualification result; the method for judging whether the call request meets the flow execution condition according to the preset flow configuration data of each calling party and obtaining the flow judgment result comprises the following steps:

obtaining the number of the calling requests of the calling party and the priority of the calling requests;

configuring the flow configuration data of the calling party through an isolation algorithm;

determining whether the number of call requests of the caller exceeds the corresponding traffic configuration data,

if yes, the calling request is represented to not meet the flow execution condition, the priority of the calling request is adjusted to be low, and a flow excess result is generated;

if not, the calling request is represented to meet the flow execution condition, and a flow qualified result is generated.

7. The method of any of claims 1-6, wherein the call configuration information includes a callback address and a timeout time; analyzing interface configuration information corresponding to the caller information according to the caller information in the calling task, and after responding to the calling request, the method further comprises the following steps:

obtaining a query response result corresponding to the calling task;

and sending the query response result to the caller through an asynchronous callback mechanism according to the callback address and the timeout time.

8. A data call apparatus, comprising:

the receiving request module is used for receiving a calling request, and the calling request comprises calling configuration information; the calling configuration information comprises calling party information and interface configuration information corresponding to the calling party information;

the task generation module is used for generating a calling task according to the calling request;

and the response module is used for analyzing the interface configuration information corresponding to the calling party information according to the calling party information in the calling task and responding to the calling request.

9. An electronic device, comprising: a processor and a memory, the memory storing machine-readable instructions executable by the processor, the machine-readable instructions, when executed by the processor, performing the method of any of claims 1 to 7.

10. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, performs the method of any one of claims 1 to 7.

Images