CN116489100A - Data processing method, device, computer equipment and storage medium - Google Patents
Data processing method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN116489100A CN116489100A CN202310460891.8A CN202310460891A CN116489100A CN 116489100 A CN116489100 A CN 116489100A CN 202310460891 A CN202310460891 A CN 202310460891A CN 116489100 A CN116489100 A CN 116489100A
- Authority
- CN
- China
- Prior art keywords
- request
- function
- token
- processed
- target function
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003860 storage Methods 0.000 title claims abstract description 14
- 238000003672 processing method Methods 0.000 title abstract description 21
- 230000006870 function Effects 0.000 claims abstract description 373
- 238000012545 processing Methods 0.000 claims abstract description 149
- 230000002159 abnormal effect Effects 0.000 claims abstract description 92
- 230000000670 limiting effect Effects 0.000 claims abstract description 41
- 238000000034 method Methods 0.000 claims abstract description 41
- 238000004590 computer program Methods 0.000 claims description 27
- 238000005111 flow chemistry technique Methods 0.000 claims description 10
- 238000012216 screening Methods 0.000 claims description 5
- 238000005516 engineering process Methods 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000004044 response Effects 0.000 description 5
- 238000001914 filtration Methods 0.000 description 4
- 238000007726 management method Methods 0.000 description 4
- 230000001960 triggered effect Effects 0.000 description 4
- 238000009826 distribution Methods 0.000 description 3
- 238000013439 planning Methods 0.000 description 3
- 238000013475 authorization Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000005304 joining Methods 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 206010033799 Paralysis Diseases 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/20—Traffic policing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q20/00—Payment architectures, schemes or protocols
- G06Q20/38—Payment protocols; Details thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/215—Flow control; Congestion control using token-bucket
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Strategic Management (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Financial Or Insurance-Related Operations Such As Payment And Settlement (AREA)
Abstract
The application relates to a data processing method, a data processing device, computer equipment and a storage medium. To the field of financial technology or other related fields. The method comprises the following steps: acquiring request control information generated based on a target function in a financial business system; the target function is a function with abnormal operation condition in the financial service system, and the request control information is used for identifying a task request belonging to the target function; receiving a task request to be processed, and determining a request processing mode corresponding to the task request to be processed according to the request control information; the request processing mode comprises a first processing mode aiming at a target function; under the condition that the request processing mode is a first processing mode, configuring an abnormal token for a task request to be processed; the exception token is used for setting the task request to be processed to be in a current limiting state. The method can realize dynamic current limiting processing aiming at service functions in the financial service system, is effectively applicable to daily processing scenes and high concurrency scenes, and plays a role in dynamic current limiting.
Description
Technical Field
The present application relates to the field of financial technology, and in particular, to a data processing method, apparatus, computer device, storage medium, and computer program product.
Background
Aiming at a financial service system, when network jitter occurs or the performance of a certain program branch does not reach the standard, the response speed of the system is reduced, the use resources of the system such as a connection pool are exhausted, further the problems of request blocking, system processing performance reduction and the like are possibly caused, even the system is paralyzed, and corresponding current limiting processing is needed.
At present, the traditional current limiting method is generally aimed at high concurrency scenes with time regularity, cannot cover most daily ordinary processing scenes, and cannot effectively conduct current limiting processing on the daily ordinary processing scenes.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a data processing method, apparatus, computer device, storage medium, and computer program product that can solve the foregoing problems.
In a first aspect, the present application provides a data processing method, the method comprising:
acquiring request control information generated based on a target function in a financial business system; the target function is a function with abnormal operation conditions in the financial service system, and the request control information is used for identifying a task request belonging to the target function;
Receiving a task request to be processed, and determining a request processing mode corresponding to the task request to be processed according to the request control information; the request processing mode comprises a first processing mode aiming at the target function;
configuring an exception token for the task request to be processed under the condition that the request processing mode is the first processing mode; the exception token is used for setting the task request to be processed to be in a current limiting state.
In one embodiment, the acquiring the request control information generated based on the target function in the financial service system includes:
acquiring functional operation data of the financial business system; the function operation data are used for representing the operation conditions of different functions in the financial service system, and the operation condition of each function is determined based on the historical operation condition and the real-time operation condition of the function;
determining a function with abnormal operation condition from a plurality of functions contained in the financial service system according to the function operation data and preset abnormal operation identification information, and taking the function as the target function;
and obtaining the request control information according to the target function.
In one embodiment, after the step of obtaining the request control information according to the target function, the method further includes:
deleting any target function from the request control information when the latest running condition of any target function is not matched with the abnormal running identification information, so as to obtain updated request control information;
and when detecting that the latest running condition of any function except the target function is matched with the abnormal running identification information, taking any function as the target function, adding the request control information, and obtaining updated request control information.
In one embodiment, the determining, according to the function operation data and the preset abnormal operation identification information, a function with an abnormal operation condition from a plurality of functions included in the financial service system, as the target function, includes:
identifying a function in a slow running state from the plurality of functions as the target function according to the function running data and preset abnormal running identification information; the function in the slow running state has a task request to which the running is finished;
Identifying a function in a dead running state from the plurality of functions as the target function according to the function running data; the function in the dead running state does not have a task request to which the running is finished.
In one embodiment, the abnormal operation identification information includes a plurality of abnormal operation indexes, and the identifying a function in a slow operation state from the plurality of functions according to the function operation data and preset abnormal operation identification information includes:
screening out a function with a task request of which the operation is finished from the functions, and obtaining operation index statistical information corresponding to the function according to the function operation data;
and under the condition that the operation index statistical information accords with the abnormal operation indexes, determining the function to be in a slow operation state.
In one embodiment, the request processing means further includes a second processing means for a function other than the target function, and the method further includes:
under the condition that the request processing mode is the second processing mode, configuring a normal token for the task request to be processed; the normal token is used for setting the task request to be processed to be in a normal flow processing state;
The method further comprises the steps of:
dividing system use resources of the financial service system to obtain first system use resources and second system use resources;
and performing flow limiting processing on the request configured with the abnormal token through the first system using the resource, and performing normal flow processing on the request configured with the normal token through the second system using the resource.
In one embodiment, the target function includes a function in a slow running state, the exception token includes a slow running token, the configuring the exception token for the pending task request includes:
for the function in the slow running state, distributing the slow running token to be used to the task request to be processed when the existence of the slow running token to be used is detected; the slow running token is used for limiting the request processing of the task request to be processed;
and ending the request processing of the task request to be processed when detecting that no slow running token to be used exists.
In one embodiment, the target function includes a function in a stalled-operation state, the exception token includes a stalled-operation token, and the configuring the exception token for the pending task request includes:
Distributing the stagnant operation token to the task request to be processed aiming at the function in the stagnant operation state; the stagnation running token is used for ending the request processing of the task request to be processed.
In a second aspect, the present application also provides a data processing apparatus, the apparatus comprising:
the request control information acquisition module is used for acquiring request control information generated based on a target function in the financial business system; the target function is a function with abnormal operation conditions in the financial service system, and the request control information is used for identifying a task request belonging to the target function;
the request processing mode determining module is used for receiving a task request to be processed and determining a request processing mode corresponding to the task request to be processed according to the request control information; the request processing mode comprises a first processing mode aiming at the target function;
the token configuration module is used for configuring an abnormal token for the task to be processed request under the condition that the request processing mode is the first processing mode; the exception token is used for setting the task request to be processed to be in a current limiting state.
In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the steps of the data processing method as described above when the processor executes the computer program.
In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the data processing method as described above.
In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when being executed by a processor, implements the steps of the data processing method as described above.
According to the data processing method, the device, the computer equipment, the storage medium and the computer program product, the request control information generated based on the target function in the financial service system is acquired, the target function is a function with abnormal operation conditions in the financial service system, the request control information is used for identifying task requests belonging to the target function, then the task requests to be processed are received, a request processing mode corresponding to the task requests to be processed is determined according to the request control information, the request processing mode comprises a first processing mode aiming at the target function, further, under the condition that the request processing mode is the first processing mode, an abnormal token is configured for the task requests to be processed, the abnormal token is used for setting the task requests to be processed to be in a current limiting state, dynamic current limiting processing of the service function in the financial service system is realized, the task requests belonging to the function with abnormal operation conditions can be identified to be filtered and distributed according to the request control information, the task requests with the abnormal operation conditions can be effectively applied to daily common processing scenes and high concurrency scenes, dynamic use and planning of system resources are performed, and the dynamic current limiting effect is achieved.
Drawings
FIG. 1 is a flow chart of a data processing method according to an embodiment;
FIG. 2 is a schematic diagram of a dynamic current limit process flow in one embodiment;
FIG. 3 is a schematic diagram of a request filtering distribution flow in one embodiment;
FIG. 4 is a schematic diagram of a dynamic generation flow of request control information in one embodiment;
FIG. 5 is a flow chart of another data processing method according to an embodiment;
FIG. 6 is a block diagram of a data processing apparatus in one embodiment;
FIG. 7 is an internal block diagram of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for presentation, analyzed data, etc.) related in the present application are both information and data authorized by the user or sufficiently authorized by each party; correspondingly, the application also provides a corresponding user authorization entry for the user to select authorization or select rejection.
In one embodiment, as shown in fig. 1, a data processing method is provided, where this embodiment is applied to a terminal to illustrate the method, and it is understood that the method may also be applied to a server, and may also be applied to a system including a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the steps of:
step 101, acquiring request control information generated based on a target function in a financial business system;
the target function may be a function in which an abnormal operation condition exists in the financial service system, and the financial service system may have a plurality of service functions, such as a function for logging in a service, a function for exporting data from a service, and the like, which is not particularly limited in the embodiment; the function with abnormal operation condition can be determined from a plurality of service functions according to the preset slow transaction characteristics and used as a target function, such as slow transaction identification.
As an example, the request control information may be used to identify task requests pertaining to a target function, e.g., the request control information may be a transaction control list, which may include one or more slow transactions identified.
In practical application, by acquiring function operation data of the financial service system, the operation conditions of different functions in the financial service system can be obtained based on the historical operation conditions and the real-time operation conditions of each function in the financial service system, then the function with the abnormal operation condition can be determined from a plurality of functions contained in the financial service system by combining preset abnormal operation identification information and the operation conditions of different functions to serve as a target function, and further request control information such as a transaction control list can be generated based on the target function.
Specifically, historical operating conditions and real-time operating conditions in the financial business system can be collected, transactions conforming to slow transaction characteristics (i.e., functions with abnormal operating conditions) can be identified by combining preset slow transaction characteristics, and can be included in a transaction control list (i.e., request control information).
In an example, the target function may include a function in a slow running state, such as a task request to which it belongs ending in running with a long time; the system can also comprise a function in a dead running state, such as that the task request to which the function belongs is in the dead running state after running starts, and no record of running end exists.
In yet another example, as shown in FIG. 2, by identifying and managing slow functions, transactions that are in accordance with slow transaction characteristics (i.e., functions that are in a slow running state) may be dynamically added to the control list (i.e., request control information), and the control list may be dynamically exited for transactions that are no longer in accordance with slow transaction characteristics; the control list can also be dynamically added or exited for the detected function in the dead running state. Therefore, the request control information can be dynamically updated to realize the dynamic current limiting effect.
102, receiving a task request to be processed, and determining a request processing mode corresponding to the task request to be processed according to the request control information;
the request processing means may include a first processing means for a target function and a second processing means for a function other than the target function.
In a specific implementation, as shown in fig. 2, when the financial service system receives a transaction request (i.e., a task request to be processed), the financial service system may determine based on a control list (i.e., request control information) obtained by identifying and managing slow transactions, and by identifying a task request belonging to a target function, the received task request to be processed may be filtered and distributed, so that a request processing manner corresponding to the task request to be processed may be determined based on whether the function belongs to the function to which the task request to be processed belongs and based on whether the function exists in the request control information.
For example, as shown in fig. 3, when a task request to be processed (such as a transaction request in fig. 3) is received, whether a function to which the task request to be processed belongs exists in the request control information (such as a control list in fig. 3) can be identified according to the request control information, so as to determine a request processing manner for the task request to be processed.
In an example, as shown in fig. 2, the received task request to be processed is filtered and distributed, so that an identification output result can be obtained, for example, a request belonging to a normal transaction, a request belonging to a slow transaction (i.e., a function in a slow running state), and a request belonging to a dead transaction (i.e., a function in a dead running state) can be distinguished according to request control information.
Step 103, configuring an exception token for the task request to be processed under the condition that the request processing mode is the first processing mode; the exception token is used for setting the task request to be processed to be in a current limiting state.
After determining a request processing mode, configuring a normal token for a task request to be processed, wherein the function of the normal token is not in the request control information; for a task request to be processed, where the function is in the request control information, an exception token may be configured for the task request to be processed, as shown in fig. 2, and according to the output result, transaction processing may be performed on the task request to be processed based on the correspondingly configured token.
In an example, as shown in fig. 3, for a request belonging to a normal transaction, the subsequent processing can be directly performed without current limit token management; for the request belonging to slow transaction, the request can be processed by a token bucket algorithm and then is continuously processed or refused; for requests belonging to stagnant transactions, a stagnant run token may be acquired, ending the process directly.
In an alternative embodiment, after the request response is completed, the token may be put back into the token bucket, and token management may be performed as follows:
1. initializing a token: according to the system resource allocation condition, a plurality of tokens can be initialized; upon detecting an exception to the token process, the token may be triggered to reinitialize.
2. Token checking: by the background service, whether a token which is not released for a long time exists in the currently used tokens or not can be checked regularly, and further, the token can be triggered to be reinitialized under a certain condition.
3. Obtaining a token: when a request arrives, it may be configured with an unused token and the token holder may be registered as the current request id and the start time of the token hold may be recorded and the token status may be updated to used. The request handler may record the currently used token number so that the token is returned when the request processing is completed.
4. Put back the token: when the request response is completed, checking whether the holder of the token in the token bucket is the current request id and whether the state is used, and if the check is passed, updating the state of the token to be unused; if the check fails, the token status may not be updated, as some requests do not put back tokens for a long time, and after the check finds an exception by the token management process, the token is triggered to be reinitialized.
According to the data processing method, the request control information generated based on the target function in the financial service system is acquired, the target function is a function with abnormal operation conditions in the financial service system, the request control information is used for identifying task requests belonging to the target function, then the task requests to be processed are received, a request processing mode corresponding to the task requests to be processed is determined according to the request control information, the request processing mode comprises a first processing mode aiming at the target function, further, under the condition that the request processing mode is the first processing mode, an abnormal token is configured for the task requests to be processed, the abnormal token is used for setting the task requests to be processed to be in a current limiting state, dynamic current limiting processing of the service function in the financial service system is achieved, the task requests which are received can be filtered and distributed according to the request control information, the task requests which are subjected to the function with the abnormal operation conditions are identified, the task requests which are subjected to current limiting processing can be effectively applied to daily common processing scenes and high concurrency scenes, and dynamic use and planning of system resources are achieved.
In one embodiment, the acquiring the request control information generated based on the target function in the financial service system may include the steps of:
acquiring functional operation data of the financial business system; the function operation data are used for representing the operation conditions of different functions in the financial service system, and the operation condition of each function is determined based on the historical operation condition and the real-time operation condition of the function; determining a function with abnormal operation condition from a plurality of functions contained in the financial service system according to the function operation data and preset abnormal operation identification information, and taking the function as the target function; and obtaining the request control information according to the target function.
In practical application, the dynamic current limiting can be adjusted based on the real-time historical operation condition and the historical operation condition of each function to judge whether to enter or exit the control list, and the functions with abnormal operation conditions can be identified by collecting the function operation data of the financial service system and judging by combining with the preset slow transaction characteristics (namely abnormal operation identification information), and can be incorporated into the request control information.
Specifically, during the process of collecting the historical operating condition and the real-time operating condition, a function detail registration queue, a statistics index result queue and a system resource may be initialized, wherein the function detail registration queue may include a start registration queue and an end registration queue.
In an example, the initial registration queue may be used for writing when the function is triggered to record function association information, for example, when a request of a certain function is received, corresponding function name, ID, service code, dependency relationship between functions (for example, if the precondition that the function C operates is that the function B is required to operate first) and the like may be recorded; the end register queue may be used for writing after the function belongs to the request operation is ended, so as to record processing time consumption and resource occupation conditions (such as use conditions of memory, thread pool, CPU and the like).
In yet another example, the statistics result queue may be used to store statistics, for example, statistics results within a preset time range (such as the last 10 seconds) of the function may be calculated using overall statistics (such as processing time, resource occupation, etc.) of the function for the function to which the response completion request belongs according to the ending registration queue.
Compared with the traditional method, the technical scheme of the embodiment can realize dynamic current limiting by combining a token bucket algorithm and a queue mode, can identify a slow request according to a control list dynamically generated by a system by filtering and distributing the request, can isolate the slow request, and can distribute limited system resources to the slow request by using the token bucket algorithm, thereby achieving the effect of dynamic current limiting.
In this embodiment, by acquiring function operation data of the financial service system, and then determining a function with an abnormal operation condition from a plurality of functions included in the financial service system according to the function operation data and preset abnormal operation identification information, the function is used as a target function, and further request control information is obtained according to the target function, the function with the abnormal operation condition can be dynamically identified according to real-time and historical operation conditions, so that dynamic planning and use of system resources can be performed, and most of daily use scenes can be covered besides high concurrency scenes.
In one embodiment, after the step of obtaining the request control information according to the target function, the method may further include the steps of:
Deleting any target function from the request control information when the latest running condition of any target function is not matched with the abnormal running identification information, so as to obtain updated request control information; and when detecting that the latest running condition of any function except the target function is matched with the abnormal running identification information, taking any function as the target function, adding the request control information, and obtaining updated request control information.
In a specific implementation, as shown in fig. 4, by collecting the historical running conditions and the real-time running conditions of different functions in the system, the functions of abnormal running conditions can be identified and managed, and the control list can be dynamically exited for the target functions which do not conform to the slow transaction characteristics any more (i.e. are not matched with the abnormal running identification information), namely any target function is deleted from the request control information, so that updated request control information is obtained; the control list can be dynamically added for the newly detected function which accords with the slow transaction characteristic (namely, is matched with the abnormal operation identification information), namely, any function is taken as a target function, the request control information is added, and the updated request control information is obtained, so that the control list can be dynamically updated.
In this embodiment, when the latest running condition of any one target function is not matched with the abnormal running identification information, any one target function is deleted from the request control information to obtain updated request control information, and when the latest running condition of any one function other than the target function is detected to be matched with the abnormal running identification information, any one function is used as the target function to add the request control information to obtain updated request control information, so that the request control information can be dynamically updated to realize the dynamic current limiting effect.
In one embodiment, the determining, according to the function operation data and the preset abnormal operation identification information, a function with an abnormal operation condition from a plurality of functions included in the financial service system, as the target function may include the following steps:
identifying a function in a slow running state from the plurality of functions as the target function according to the function running data and preset abnormal running identification information; the function in the slow running state has a task request to which the running is finished; identifying a function in a dead running state from the plurality of functions as the target function according to the function running data; the function in the dead running state does not have a task request to which the running is finished.
In an example, a function in a slow running state may be identified as a target function to write into the control list according to the statistics index result queue and the slow transaction characteristics (i.e., function running data and preset abnormal running identification information); the control list may be written as a target function to a function to which a request having a record in the start registration queue but not having a record in the end registration queue belongs, and a function determined to be in a dead operation state due to its incomplete operation.
In yet another example, time-consuming statistics may be performed regularly, by writing a request for completion of a response to an end registration queue, according to a function list, statistics index results corresponding to different functions may be calculated according to data such as processing time consumption and resource occupation recorded in the end registration queue, and registered to the statistics index result queue, and further the end registration queue may be emptied through a regular backup process for the statistics index result queue, for example, resource usage of a previous day system may be acquired at a preset time point (e.g., 6 a day in the morning) for token bucket segmentation during the current limiting process.
In this embodiment, by identifying, from the plurality of functions, a function in a slow running state as a target function according to the function running data and preset abnormal running identification information, and identifying, from the plurality of functions, a function in a dead running state as a target function according to the function running data, a function in an abnormal running condition can be effectively identified, and data support is provided for subsequent request filtering distribution processing.
In one embodiment, the abnormal operation identification information includes a plurality of abnormal operation indexes, and the identifying a function in a slow operation state from the plurality of functions according to the function operation data and preset abnormal operation identification information may include the steps of:
screening out a function with a task request of which the operation is finished from the functions, and obtaining operation index statistical information corresponding to the function according to the function operation data; and under the condition that the operation index statistical information accords with the abnormal operation indexes, determining the function to be in a slow operation state.
In practical applications, statistical indicators for slow-transaction characteristics include, but are not limited to, the following indicators (i.e., a plurality of abnormal operation indicators):
TiMES of the last 10 seconds of a certain transaction;
a total time of day COST of the last 10 seconds of a transaction;
the number pool=tips/1000 of connection pools used per second for the last 10 seconds of a transaction;
the last 10 seconds of a transaction takes more than 3 seconds;
time consumption rate of a certain transaction = (Pool-T0)/T0, T0 is the number of connection pools used per second on average of history.
In an example, a function having a task request to which operation ends may be screened from a plurality of functions according to a statistics index result queue, and a statistics index result (i.e., operation index statistics information) of each function may be obtained, so that for the screened function, a determination may be made based on the statistics index of the slow transaction feature, and the function conforming to the slow transaction feature may be determined as a function in a slow operation state, so as to dynamically add to the control list.
In this embodiment, the function having the task request of the operation end is screened from the plurality of functions, and the operation index statistical information corresponding to the function is obtained according to the function operation data, so that the function is determined to be in the slow operation state under the condition that the operation index statistical information accords with a plurality of abnormal operation indexes, and the slow operation function can be effectively identified based on a plurality of abnormal operation indexes which are flexibly set.
In one embodiment, the request processing manner may further include a second processing manner for a function other than the target function, and may further include the steps of:
under the condition that the request processing mode is the second processing mode, configuring a normal token for the task request to be processed; the normal token is used for setting the task request to be processed to be in a normal flow processing state;
in a specific implementation, after filtering and distributing, a normal token can be acquired for a request belonging to a normal transaction so as to directly carry out subsequent processing without current-limiting token management.
The method further comprises the steps of:
dividing system use resources of the financial service system to obtain first system use resources and second system use resources; and performing flow limiting processing on the request configured with the abnormal token through the first system using the resource, and performing normal flow processing on the request configured with the normal token through the second system using the resource.
In one example, to distinguish between a limited transaction and a normal transaction during a limited process, system resources may be partitioned, and by initializing system usage resources for normal and limited transactions, first system usage resources for processing limited transactions may be obtained, and second system usage resources for processing normal transactions.
For example, the system memory is 16G, the maximum connection number of the thread pool is 100, the resource division can be performed according to the ratio of 2:8, the memory can be configured to be 3.2G for the current limiting transaction, the maximum connection number of the thread pool is 20, the memory can be configured to be 12.8G for the normal transaction, and the maximum connection number of the thread pool is 80.
In this embodiment, by dividing the system usage resources of the financial service system to obtain the first system usage resources and the second system usage resources, further performing the current limiting processing on the request configured with the abnormal token through the first system usage resources, and performing the normal flow processing on the request configured with the normal token through the second system usage resources, the system resources can be dynamically used and planned, and the dynamic current limiting effect is achieved.
In one embodiment, the target function may include a function in a slow running state, the exception token may include a slow running token, and the configuring the exception token for the task to be processed request may include the steps of:
For the function in the slow running state, distributing the slow running token to be used to the task request to be processed when the existence of the slow running token to be used is detected; the slow running token is used for limiting the request processing of the task request to be processed; and ending the request processing of the task request to be processed when detecting that no slow running token to be used exists.
In practical application, as shown in fig. 3, for a request belonging to a slow transaction, that is, a request belonging to a function in a slow running state, the request may be processed by a token bucket algorithm and then subjected to subsequent processing or refused, for example, a slow running token may be acquired first and then distributed to subsequent processing, and if the slow running token cannot be acquired or is issued, that is, there is no slow running token to be used, the request may be directly refused.
In this embodiment, by distributing the slow running token to be used to the task request to be processed when the slow running token to be used is detected to exist for the function in the slow running state, and ending the request processing of the task request to be processed when the slow running token to be used is detected not to exist, the current limiting processing can be effectively performed on the function in the slow running state, and the system processing efficiency is improved.
In one embodiment, the target function may include a function in a stalled running state, the exception token may include a stalled running token, and the configuring the exception token for the pending task request may include the steps of:
distributing the stagnant operation token to the task request to be processed aiming at the function in the stagnant operation state; the stagnation running token is used for ending the request processing of the task request to be processed.
In an example, a stalled-operation token may be obtained for a request belonging to a stalled transaction, i.e., a request belonging to a function in a stalled-operation state, directly ending the process.
In this embodiment, by distributing the stagnant operation token to the task request to be processed for the function in the stagnant operation state to end the request processing of the task request to be processed, the function in the stagnant operation state can be prevented from occupying system resources, and the system processing efficiency is improved.
In one embodiment, as shown in FIG. 5, a flow diagram of another data processing method is provided. In this embodiment, the method includes the steps of:
in step 501, obtaining functional operation data of a financial service system; the function operation data is used for representing the operation conditions of different functions in the financial service system, and the operation condition of each function is determined based on the historical operation condition and the real-time operation condition of the function. In step 502, according to the function operation data and the preset abnormal operation identification information, a function with abnormal operation condition is determined from a plurality of functions included in the financial service system, and the function is taken as a target function, and request control information is obtained according to the target function. In step 503, a task request to be processed is received, and a request processing manner corresponding to the task request to be processed is determined according to the request control information. In step 504, if the request processing mode is the first processing mode, configuring an exception token for the task request to be processed; the target functions include a function in a slow running state and a function in a dead running state, and the abnormal tokens include a slow running token and a dead running token. In step 505, in the case where the request processing manner is the second processing manner, a normal token is configured for the task request to be processed. In step 506, the system usage resources of the financial business system are partitioned to obtain first system usage resources and second system usage resources. In step 507, the request configured with the abnormal token is subject to flow-limiting processing by the first system using the resource, and the request configured with the normal token is subject to normal flow processing by the second system using the resource. It should be noted that, the specific limitation of the above steps may be referred to the specific limitation of a data processing method, which is not described herein.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides a data processing device for realizing the above related data processing method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation of one or more embodiments of the data processing device provided below may refer to the limitation of the data processing method hereinabove, and will not be repeated herein.
In one embodiment, as shown in FIG. 6, there is provided a data processing apparatus comprising:
a request control information acquisition module 601, configured to acquire request control information generated based on a target function in a financial service system; the target function is a function with abnormal operation conditions in the financial service system, and the request control information is used for identifying a task request belonging to the target function;
the request processing mode determining module 602 is configured to receive a task request to be processed, and determine a request processing mode corresponding to the task request to be processed according to the request control information; the request processing mode comprises a first processing mode aiming at the target function;
a token configuration module 603, configured to configure an exception token for the task to be processed request, where the request processing manner is the first processing manner; the exception token is used for setting the task request to be processed to be in a current limiting state.
In one embodiment, the request control information acquisition module 601 includes:
the function operation data acquisition sub-module is used for acquiring the function operation data of the financial business system; the function operation data are used for representing the operation conditions of different functions in the financial service system, and the operation condition of each function is determined based on the historical operation condition and the real-time operation condition of the function;
A target function determining sub-module, configured to determine, according to the function operation data and preset abnormal operation identification information, a function having an abnormal operation condition from a plurality of functions included in the financial service system, as the target function;
and the request control information obtaining sub-module is used for obtaining the request control information according to the target function.
In one embodiment, the apparatus further comprises:
the deleting module is used for deleting any target function from the request control information when the latest running condition of any target function is not matched with the abnormal running identification information, so as to obtain updated request control information;
and the joining module is used for joining the request control information by taking any function as the target function when detecting that the latest running condition of any function except the target function is matched with the abnormal running identification information, so as to obtain updated request control information.
In one embodiment, the target function determination submodule includes:
a slow function determining unit configured to identify a function in a slow running state from the plurality of functions as the target function according to the function running data and preset abnormal running identification information; the function in the slow running state has a task request to which the running is finished;
A dead function determining unit configured to identify, as the target function, a function in a dead operation state from the plurality of functions based on the function operation data; the function in the dead running state does not have a task request to which the running is finished.
In one embodiment, the abnormal operation identification information includes a plurality of abnormal operation indexes, and the slow function determination unit includes:
the statistical information obtaining subunit is used for screening out the function with the task request of the operation end from the plurality of functions and obtaining operation index statistical information corresponding to the function according to the function operation data;
and the slow function screening subunit is used for determining the function to be in a slow running state under the condition that the running index statistical information accords with the plurality of abnormal running indexes.
In one embodiment, the request processing means further includes second processing means for a function other than the target function, and the apparatus further includes:
the normal processing module is used for configuring a normal token for the task request to be processed under the condition that the request processing mode is the second processing mode; the normal token is used for setting the task request to be processed to be in a normal flow processing state;
The apparatus further comprises:
the resource segmentation module is used for segmenting the system use resources of the financial service system to obtain first system use resources and second system use resources;
and the resource distinguishing processing module is used for carrying out flow limiting processing on the request configured with the abnormal token through the first system using the resource and carrying out normal flow processing on the request configured with the normal token through the second system using the resource.
In one embodiment, the target function includes a function in a slow running state, the exception token includes a slow running token, and the token configuration module 603 includes:
the slow token distribution sub-module is used for distributing the slow running tokens to be used to the task request to be processed when the slow running tokens to be used are detected to exist for the functions in the slow running state; the slow running token is used for limiting the request processing of the task request to be processed;
and the ending processing sub-module is used for ending the request processing of the task request to be processed when detecting that the slow running token to be used does not exist.
In one embodiment, the target function includes a function in a dead-run state, the exception token includes a dead-run token, and the token configuration module 603 includes:
A stagnation token distributing sub-module, configured to distribute, for a function in a stagnation running state, the stagnation running token to the task request to be processed; the stagnation running token is used for ending the request processing of the task request to be processed.
Each of the modules in the above-described data processing apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a terminal, and the internal structure of which may be as shown in fig. 7. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media.
It will be appreciated by those skilled in the art that the structure shown in fig. 7 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of:
acquiring request control information generated based on a target function in a financial business system; the target function is a function with abnormal operation conditions in the financial service system, and the request control information is used for identifying a task request belonging to the target function;
receiving a task request to be processed, and determining a request processing mode corresponding to the task request to be processed according to the request control information; the request processing mode comprises a first processing mode aiming at the target function;
configuring an exception token for the task request to be processed under the condition that the request processing mode is the first processing mode; the exception token is used for setting the task request to be processed to be in a current limiting state.
In an embodiment, the processor, when executing the computer program, also implements the steps of the data processing method in the other embodiments described above.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring request control information generated based on a target function in a financial business system; the target function is a function with abnormal operation conditions in the financial service system, and the request control information is used for identifying a task request belonging to the target function;
receiving a task request to be processed, and determining a request processing mode corresponding to the task request to be processed according to the request control information; the request processing mode comprises a first processing mode aiming at the target function;
configuring an exception token for the task request to be processed under the condition that the request processing mode is the first processing mode; the exception token is used for setting the task request to be processed to be in a current limiting state.
In an embodiment, the computer program, when executed by a processor, also implements the steps of the data processing method in the other embodiments described above.
In one embodiment, a computer program product is provided comprising a computer program which, when executed by a processor, performs the steps of:
acquiring request control information generated based on a target function in a financial business system; the target function is a function with abnormal operation conditions in the financial service system, and the request control information is used for identifying a task request belonging to the target function;
receiving a task request to be processed, and determining a request processing mode corresponding to the task request to be processed according to the request control information; the request processing mode comprises a first processing mode aiming at the target function;
configuring an exception token for the task request to be processed under the condition that the request processing mode is the first processing mode; the exception token is used for setting the task request to be processed to be in a current limiting state.
In an embodiment, the computer program, when executed by a processor, also implements the steps of the data processing method in the other embodiments described above.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples only represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.
Claims (12)
1. A method of data processing, the method comprising:
acquiring request control information generated based on a target function in a financial business system; the target function is a function with abnormal operation conditions in the financial service system, and the request control information is used for identifying a task request belonging to the target function;
receiving a task request to be processed, and determining a request processing mode corresponding to the task request to be processed according to the request control information; the request processing mode comprises a first processing mode aiming at the target function;
Configuring an exception token for the task request to be processed under the condition that the request processing mode is the first processing mode; the exception token is used for setting the task request to be processed to be in a current limiting state.
2. The method of claim 1, wherein the obtaining the request control information generated based on the target function in the financial transaction system comprises:
acquiring functional operation data of the financial business system; the function operation data are used for representing the operation conditions of different functions in the financial service system, and the operation condition of each function is determined based on the historical operation condition and the real-time operation condition of the function;
determining a function with abnormal operation condition from a plurality of functions contained in the financial service system according to the function operation data and preset abnormal operation identification information, and taking the function as the target function;
and obtaining the request control information according to the target function.
3. The method according to claim 2, wherein after the step of obtaining the request control information according to the target function, the method further comprises:
deleting any target function from the request control information when the latest running condition of any target function is not matched with the abnormal running identification information, so as to obtain updated request control information;
And when detecting that the latest running condition of any function except the target function is matched with the abnormal running identification information, taking any function as the target function, adding the request control information, and obtaining updated request control information.
4. The method according to claim 2, wherein the determining, from a plurality of functions included in the financial service system, a function in which an abnormal operation condition exists as the target function, based on the function operation data and preset abnormal operation identification information, includes:
identifying a function in a slow running state from the plurality of functions as the target function according to the function running data and preset abnormal running identification information; the function in the slow running state has a task request to which the running is finished;
identifying a function in a dead running state from the plurality of functions as the target function according to the function running data; the function in the dead running state does not have a task request to which the running is finished.
5. The method according to claim 4, wherein the abnormal operation identification information includes a plurality of abnormal operation indexes, and the identifying the function in the slow operation state from the plurality of functions based on the function operation data and the preset abnormal operation identification information includes:
Screening out a function with a task request of which the operation is finished from the functions, and obtaining operation index statistical information corresponding to the function according to the function operation data;
and under the condition that the operation index statistical information accords with the abnormal operation indexes, determining the function to be in a slow operation state.
6. The method of claim 1, wherein the request processing means further comprises a second processing means for a function other than the target function, the method further comprising:
under the condition that the request processing mode is the second processing mode, configuring a normal token for the task request to be processed; the normal token is used for setting the task request to be processed to be in a normal flow processing state;
the method further comprises the steps of:
dividing system use resources of the financial service system to obtain first system use resources and second system use resources;
and performing flow limiting processing on the request configured with the abnormal token through the first system using the resource, and performing normal flow processing on the request configured with the normal token through the second system using the resource.
7. The method of any of claims 1 to 6, wherein the target function comprises a function in a slow running state, the exception token comprises a slow running token, the configuring an exception token for the pending task request comprises:
for the function in the slow running state, distributing the slow running token to be used to the task request to be processed when the existence of the slow running token to be used is detected; the slow running token is used for limiting the request processing of the task request to be processed;
and ending the request processing of the task request to be processed when detecting that no slow running token to be used exists.
8. The method of any of claims 1 to 6, wherein the target function comprises a function in a stalled-operation state, the exception token comprises a stalled-operation token, the configuring an exception token for the pending task request comprises:
distributing the stagnant operation token to the task request to be processed aiming at the function in the stagnant operation state; the stagnation running token is used for ending the request processing of the task request to be processed.
9. A data processing apparatus, the apparatus comprising:
the request control information acquisition module is used for acquiring request control information generated based on a target function in the financial business system; the target function is a function with abnormal operation conditions in the financial service system, and the request control information is used for identifying a task request belonging to the target function;
the request processing mode determining module is used for receiving a task request to be processed and determining a request processing mode corresponding to the task request to be processed according to the request control information; the request processing mode comprises a first processing mode aiming at the target function;
the token configuration module is used for configuring an abnormal token for the task to be processed request under the condition that the request processing mode is the first processing mode; the exception token is used for setting the task request to be processed to be in a current limiting state.
10. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 8 when the computer program is executed.
11. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 8.
12. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the steps of the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310460891.8A CN116489100A (en) | 2023-04-26 | 2023-04-26 | Data processing method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310460891.8A CN116489100A (en) | 2023-04-26 | 2023-04-26 | Data processing method, device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116489100A true CN116489100A (en) | 2023-07-25 |
Family
ID=87219104
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310460891.8A Pending CN116489100A (en) | 2023-04-26 | 2023-04-26 | Data processing method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116489100A (en) |
-
2023
- 2023-04-26 CN CN202310460891.8A patent/CN116489100A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20080313502A1 (en) | Systems, methods and computer products for trace capability per work unit | |
| US8631280B2 (en) | Method of measuring and diagnosing misbehaviors of software components and resources | |
| CN109471718B (en) | Computing resource configuration method, device, equipment and medium based on face recognition | |
| CN109885565B (en) | Data table cleaning method and device | |
| CN110852559A (en) | Resource allocation method and device, storage medium and electronic device | |
| US20070088871A1 (en) | Implementation of shared and persistent job queues | |
| CN110321364B (en) | Transaction data query method, device and terminal of credit card management system | |
| US8924343B2 (en) | Method and system for using confidence factors in forming a system | |
| CN111767337B (en) | Block verification method, device and equipment | |
| Adzic et al. | Data warehouse population platform | |
| CN108664322A (en) | Data processing method and system | |
| CN116489100A (en) | Data processing method, device, computer equipment and storage medium | |
| CN111857670B (en) | Application architecture determining method and device | |
| CN113269554A (en) | Data comparison method, system and storage medium | |
| CN114218303A (en) | Transaction data processing system, processing method, medium and equipment | |
| CN111694835B (en) | Number section access method, system, equipment and storage medium of logistics electronic bill | |
| CN109409843B (en) | Subject accounting method and subject accounting equipment | |
| CN112685334A (en) | Method, device and storage medium for block caching of data | |
| US9286112B2 (en) | Determining and dispatching processing agent for transaction processing | |
| CN111124854A (en) | Method, system, terminal and storage medium for distributing smoking test cases | |
| CN112699124B (en) | Circulating storage method and system based on distributed multi-storage service | |
| CN110209853A (en) | Image searching method, device and the equipment of vehicle | |
| CN116471213B (en) | Link tracking method, link tracking system and medium | |
| CN113256270B (en) | Data processing method, data processing device, data processing apparatus, and storage medium | |
| CN117290345A (en) | Method, device, equipment, medium and product for data cleaning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |