WO2024087949A1 - 变更风险防控系统、方法、电子设备及存储介质 - Google Patents

变更风险防控系统、方法、电子设备及存储介质 Download PDF

Info

Publication number
WO2024087949A1
WO2024087949A1 PCT/CN2023/119954 CN2023119954W WO2024087949A1 WO 2024087949 A1 WO2024087949 A1 WO 2024087949A1 CN 2023119954 W CN2023119954 W CN 2023119954W WO 2024087949 A1 WO2024087949 A1 WO 2024087949A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
change
change event
detection
type
Prior art date
Application number
PCT/CN2023/119954
Other languages
English (en)
French (fr)
Inventor
邱硕
王月凡
俞灏宣
Original Assignee
支付宝(杭州)信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 支付宝(杭州)信息技术有限公司 filed Critical 支付宝(杭州)信息技术有限公司
Publication of WO2024087949A1 publication Critical patent/WO2024087949A1/zh

Links

Definitions

  • the embodiments of this specification relate to the field of computer technology, and in particular to change risk prevention and control systems, methods, electronic devices, and storage media.
  • the embodiments of this specification provide a change risk prevention and control system, method, electronic device and storage medium to provide systematic anomaly detection.
  • a change risk prevention and control system comprising: a rule configuration module, used to: store multiple detection rules, wherein the detection rules are obtained in a manner comprising: providing a plurality of predefined attributes for describing a change event, and receiving a detection rule configured by a first type of user and an attribute value of an attribute configuration applicable to the detection rule; a change publishing module, used to: provide the plurality of attributes, and receive a change publishing request from a second type of user, wherein the change publishing request carries a target attribute value of an attribute of a target change event by the second type of user; a change execution module, used to: execute the target change event multiple times in a preset lifecycle management order in multiple preset execution environments; wherein before each execution of the target change event, at least according to the target attribute value, a currently matched detection rule is searched out from the stored detection rules and executed to determine whether the target change event can be executed; and after each execution of the target
  • the multiple attributes of the change event include at least one of the following attributes: attributes representing basic information of the change event, the basic information including any of the following: title information, initiator information, or time information; attributes representing type information of the change event, the type information including any of the following: code release type or parameter configuration type; attributes representing execution information of the change event, the execution information including any of the following: batch release mode information, blue-green release mode information, or shutdown release mode information; attributes representing impact scope information of the change event, the impact scope information including any of the following: affected object information or impact degree information.
  • the change execution module is used to: expand one or more extended attribute values according to the target attribute value, and search for and execute the currently matched detection rule from the stored detection rules according to the target attribute value and the one or more extended attribute values; wherein the method for obtaining the extended attribute value includes: determining the upstream business system or downstream business system of the target business system, as well as other applications associated with the target application, based on the target application to be changed by the target change event and the target business system to which the target application belongs, generating The impact scope information includes the extended attribute value of any of the upstream business system, the downstream business system or the other applications.
  • the change risk prevention and control system is connected to multiple business systems, and the change events are used to change one or more applications in the business systems; the change risk prevention and control system integrates multiple detection capabilities; the rule configuration module includes any of the following sub-modules: a first configuration sub-module, used to: provide a configuration function for each of the detection capabilities, obtain the configuration parameters of the second type of users for the business systems targeted by one or more of the detection capabilities through the configuration function and generate detection rules; a second configuration sub-module, used to: provide a code configuration function, obtain the script file describing the detection rules submitted by the second type of users through the code configuration function, and generate detection rules based on the script file; a third configuration sub-module, used to: provide a service provision interface, obtain the detection rules submitted by the second type of users through the service provision interface and store them.
  • a first configuration sub-module used to: provide a configuration function for each of the detection capabilities, obtain the configuration parameters of the second type of users for the business systems targeted by one or more of the detection capabilities
  • the currently matched detection rules found before the execution of the target change event include: a detection rule for detecting whether the release conditions of the target change event are met, and the release conditions include any of the following: the condition that the risk level of the target change event is less than a preset level threshold; wherein the level threshold is related to the business volume of the target business system at the current moment, and/or is related to whether the current moment is within a specified date; the target business system is the business system changed by the target change event; or, a detection rule for detecting whether the parameter value of the target parameter configured in the target change event of the parameter configuration type passes the legality check.
  • the multiple detection rules include preset detection rules; after each execution of the change event, the currently matched detection rules are found from the stored first detection rules according to the target attribute value, including: error rules that compare the number of errors and/or error types before and after the target change event is released, and determine whether an abnormality occurs in the updated application according to the comparison results; link rules that compare the business data transmitted between the target business system and the associated business systems before and after the target change event is released, and determine whether an abnormality occurs in the updated application according to the comparison results of the business data; wherein the target business system is the business system to which the application belongs, and the associated business systems are the upstream and/or downstream business systems of the target business system.
  • the multiple preset execution environments include: a pre-release execution environment, a grayscale release environment, and a formal release environment; the preset lifecycle management order is pre-release execution environment, grayscale release environment, and formal release environment.
  • a change risk prevention and control method comprising: storing a plurality of detection rules, wherein the detection rules are obtained in the following manner: providing a plurality of predefined attributes for describing a change event, and receiving a detection rule configured by a first type of user and an attribute value of an attribute configuration applicable to the detection rule; providing the plurality of attributes, and receiving a change release request from a second type of user, the change release request carrying a target attribute value of an attribute of a target change event by the second type of user; executing the target change event multiple times in a plurality of preset execution environments in a preset lifecycle management order; wherein, before each execution of the target change event, at least according to the target attribute value, a currently matching detection rule is searched out from the stored detection rules and executed to determine whether the target change event can be executed; and after each execution of the target change event, at least according to the target attribute value, a currently matching detection rule is searched out from the stored detection rules and
  • an electronic device comprising: a processor; a memory for storing processor executable instructions; wherein the processor implements the operation of any method described in the first aspect when calling the executable instructions.
  • a computer-readable storage medium wherein a plurality of computer instructions are stored on the computer-readable storage medium, and when the computer instructions are executed, the computer instructions execute any of the first aspect. method.
  • the embodiments of the present specification provide a change risk prevention and control system, a method, an electronic device and a storage medium, a change risk prevention and control system rule configuration module, used to: store multiple detection rules, the detection rules are obtained in a manner including: providing a plurality of predefined attributes for describing a change event, and receiving a detection rule configured by a first type of user and an attribute value of an attribute configuration applicable to the detection rule; a change release module, used to: provide the plurality of attributes, and receive a change release request from a second type of user, the change release request carrying a target attribute value of an attribute of a target change event by the second type of user; a change execution module, used to: execute the target change event multiple times in a preset lifecycle management order in multiple preset execution environments; wherein, before each execution of the target change event, at least according to the target attribute value, a currently matched detection rule is searched out from the stored detection rules and executed to determine whether
  • the change risk prevention and control system provides a rule configuration module for the first type of users, and uniformly defines the attributes used to describe change events, users can configure the attribute values of the attributes and the detection rules corresponding to the attribute values, and deposit the abnormal detection experience of each first type of user into the system through the detection rules, and simply and quickly integrate the different abnormal detection experiences of each detection personnel, avoiding the situation where the abnormality cannot be detected due to the lack of experience of the detection personnel.
  • the change execution module can execute the target change event multiple times in multiple preset execution environments according to the preset life cycle management order, thereby realizing the automatic management of change events.
  • FIG. 1 is a schematic diagram of various stages of a change event according to an embodiment of the present specification.
  • FIG2 is a flow chart of a method for preventing and controlling change risks according to an embodiment of the present specification.
  • FIG3 is a flow chart of a method for preventing and controlling change risks according to another embodiment of the present specification.
  • FIG4 is a flow chart of a method for preventing and controlling change risks according to another embodiment of the present specification.
  • FIG5A is a flow chart of a method for preventing and controlling change risks according to another embodiment of the present specification.
  • FIG5B is a schematic diagram of a change risk prevention and control system according to another embodiment of the present specification.
  • FIG6 is a schematic diagram of a change risk prevention and control system according to an embodiment of the present specification.
  • FIG. 7 is a hardware structure diagram of an electronic device according to an embodiment of the present specification.
  • first, second, third, etc. may be used to describe various information in the embodiments of this specification, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other.
  • first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information.
  • word "if” as used herein may be interpreted as "at the time of” or "when” or "in response to determining”.
  • each upgrade of a business system includes changes to one or more applications.
  • the release of an application change event will update the application and upgrade the business system.
  • Application changes include the five stages shown in Figure 1.
  • the internal developer initiates an application change event request and submits the change code.
  • the reviewer browses the change code logic and approves the content of the change event based on vulnerability exceptions, code style, performance defects, duplicate code and other dimensions. After approval, it enters the simulation trial run and performs risk simulation in the test environment. Then it enters the release stage of the change event, and the change event is measured after the release.
  • the change event initiation and review stages are often achieved through manual management in the industry. That is, through the approval process of internal reviewers, the risk issues in the change process are assisted to be assessed.
  • manual management is often difficult to conduct a comprehensive assessment of risks due to the large audit throughput, omissions in the information of the change initiator, lack of experience, etc., so some potential risks cannot be discovered and resolved in the review stage.
  • the platform has also increased the ability to perceive change events.
  • the platform can actively perceive each change event and actively check the possible risks of the change event, so that reviewers can not only passively discover the risks of the change event from the information submitted by the change initiator, but also actively discover the risks of various changes during the review stage.
  • reviewers can not only passively discover the risks of the change event from the information submitted by the change initiator, but also actively discover the risks of various changes during the review stage.
  • Such a solution still cannot detect anomalies and intervene in time during the release stage of the change.
  • anomaly detection can be performed before and after the change events are released.
  • different business systems have different anomaly detection methods, and different detectors have different anomaly detection experiences.
  • anomaly detection of change events still relies on human experience. If the detector is not experienced enough, the anomaly may not be detected.
  • the detector is replaced, the previous detector can only pass on the detection experience to the next detector by word of mouth, and the detection experience cannot be precipitated in a regular manner.
  • an embodiment of this specification provides a change risk prevention and control method, which is applied to a change risk prevention and control system.
  • the change risk prevention and control system can be a change risk prevention and control system that integrates development (Development) and operations (Operations).
  • the change risk prevention and control system can be connected to one or more server clusters, and each server in the server cluster can run the same application.
  • each server in the server cluster is equipped with the same business system and therefore runs the same application.
  • the above method includes steps S11 to S13 as shown in Figure 2.
  • Step S11 storing multiple detection rules.
  • the method for acquiring the detection rule includes: providing a plurality of predefined attributes for describing a change event, and receiving a detection rule configured by a first type of user and an attribute value of an attribute configuration applicable to the detection rule.
  • Step S12 providing the plurality of attributes, and receiving a change publishing request from a second category of users.
  • the change publishing request carries the target attribute value of the attribute of the target change event for the second type of user.
  • Step S13 Execute the target change event multiple times in a plurality of preset execution environments according to a preset lifecycle management sequence.
  • the currently matching detection rule is searched out from the stored detection rules at least based on the target attribute value and executed to determine whether the target change event can be executed; and after each execution of the target change event, the currently matching detection rule is searched out from the stored detection rules at least based on the target attribute value and executed to determine whether the target change event causes an exception.
  • step S11 there is no sequential execution order between step S11 and step S12.
  • the change risk prevention and control system can be connected to multiple server clusters, and different server clusters can be equipped with different business systems. There can be interaction of business data traffic between the various business systems, or they can be independent of each other. Different domain experts, including R&D personnel, quality personnel, and operation and maintenance personnel, configure detection rules for each business system. In this way, different domain experts can configure corresponding detection rules for change events related to the business system through the rule configuration function provided by the change risk prevention and control system.
  • the change risk prevention and control system predefines attributes for describing application change events, no matter whether it is different change events of the same application, change events of different applications in the same business system, or change events of applications in different business systems, they are all described using the attributes predefine by the change risk prevention and control system.
  • a domain expert that is, the first type of user
  • he can initiate a rule configuration request to the change risk prevention and control system.
  • the change risk prevention and control system will send predefined attributes to the user.
  • the predefined attributes include multiple attributes, all the attributes are sent to the user.
  • the change risk prevention and control system can send predefined attributes to the user through a standard interface.
  • the standard interface can be SPI (Service Provider Interface).
  • the first category of users can configure attribute values for the received attributes, and configure detection rules corresponding to the attribute values.
  • the user can configure attribute values for at least some of all the received attributes, and configure at least one first category of detection rule corresponding to the attribute values.
  • the detection rules configured by the first category of users are referred to as first category detection rules; in actual applications, the detection rules stored in the system can also be implemented in other ways, for example, they are pre-configured by the technicians who developed the system of this embodiment.
  • the risk prevention and control system can send attributes 1 to 10 to the user when a setting request is made.
  • the user can only configure the attribute value A for attribute 2, and the attribute value B for attribute 3, and configure at least one corresponding first-category detection rule.
  • the user can configure one or more first-category detection rules for the attribute value A of attribute 2, and configure one or more first-category detection rules for the attribute value B of attribute 3.
  • the user can configure one or more first-category detection rules for the attribute value A of attribute 2, and the attribute value B of attribute 3.
  • the attribute value of attribute 2 of the change event is equal to A
  • the attribute value of attribute 3 is equal to B
  • one or more configured first-category detection rules are hit.
  • the change risk prevention and control system receives and stores the first type of detection rules configured by the user, and the attribute values of the attribute configuration used by the first type of detection rules.
  • the method of obtaining detection rules through the first category of users may include any of the following: providing a configuration function for each of the detection capabilities, obtaining the configuration parameters of the second category of users for the business systems targeted by one or more of the detection capabilities through the configuration function and generating detection rules; providing a code configuration function, obtaining the script file describing the detection rules submitted by the second category of users through the code configuration function, and generating detection rules based on the script file; providing a service provision interface, obtaining the detection rules submitted by the second category of users through the service provision interface and storing them.
  • the user may upload the configured attribute values and the corresponding first-category detection rules to the change risk prevention and control system in the form of a plug-in.
  • the change risk prevention and control system may provide a configuration interface to the user, and the configuration interface may display all predefined attributes.
  • the configuration interface may also include a configuration module for the user to configure attribute values and the first detection rule in the configuration module.
  • the configuration module may include an attribute configuration submodule and a rule configuration submodule. The user may configure the attribute values of at least some attributes in the attribute configuration submodule, and configure the first type of detection rule in the rule configuration submodule.
  • the change risk prevention and control system stores the first type of detection rules and the mapping relationship between the first type of detection rules and the attribute values.
  • the mapping relationship between the first type of detection rules and the attribute values can be stored in the form of an expression.
  • the expression can be an equation, one side of which can record the attribute value of the attribute configured by the user, and the other side can record the first type of detection rule or the unique identifier of the first type of detection rule. Using the unique identifier, the corresponding first type of detection rule can be read.
  • change risk prevention and control system also provides a change event publishing function, which R&D personnel can use to publish change events for applications.
  • the rule configuration function and the change event publishing function of the change risk prevention and control system can be called by the same user or by different users.
  • a second type of user can also publish a change publishing request for a change event.
  • a change event published by a second type of user is called a target change event.
  • the second type of users in this embodiment include a type of users with change requirements. Since the change risk prevention and control system predefines attributes for describing application change events, when the R&D personnel initiate a publishing request for the target change event of the application to the change risk prevention and control system, the publishing request carries the target attribute value of the attribute of the target change event.
  • the predefined attributes include multiple ones, the target attribute values of at least some of the attributes are configured by the R&D personnel, and the target attribute values of the remaining attributes are generated by the change risk prevention and control system.
  • the target change event can be executed multiple times in multiple preset execution environments according to the preset lifecycle management order; wherein, before each execution of the target change event, at least according to the target attribute value, the currently matched detection rule is searched from the stored detection rules and executed to determine whether the target change event can be executed. and after each execution of the target change event, searching for the currently matched detection rule from the stored detection rules at least according to the target attribute value and executing it to determine whether the target change event causes an exception.
  • the multiple preset execution environments include: a pre-release execution environment, a grayscale release environment, and a formal release environment; the preset lifecycle management order is the pre-release execution environment, the grayscale release environment, and the formal release environment.
  • the pre-launch execution environment refers to a pre-established test environment with multiple test servers for testing change events.
  • the test servers do not provide services for real users, but for virtual users. By making changes in the pre-launch execution environment, change events can be tested without disturbing real users.
  • a grayscale release environment is a release environment composed of dedicated grayscale servers that provide services to test users, who are selected from a large number of real users. By making changes in a grayscale release environment, a small number of test users can be used to test whether the changed business system is abnormal without disturbing a large number of users.
  • the official release environment refers to the release environment composed of all servers, which are used to provide services to all users. After the change event is confirmed to be normal through the pre-release execution environment and the gray release environment, the change event can be released in the official release environment.
  • the change risk prevention and control system in response to the above release request, can publish the target change event to the target server of the server cluster.
  • the so-called release of the target change event can be understood as updating the application corresponding to the target change event running in the target server.
  • the target first detection rule can be determined from the stored rules according to the target attribute value of the attribute carried by the release request, and the updated application can be detected for anomalies according to the target first detection rule.
  • the target first detection rule can be the first detection rule that the user has historically configured according to the above method and stored in the change risk prevention and control system.
  • the target first detection rule may be determined based on the target attribute value and the mapping relationship between the first detection rule and the attribute value stored in the above-mentioned change risk prevention and control system, and there may be one or more detections matched each time.
  • all expressions in the change risk prevention and control system may be traversed to determine all first-category detection rules that hit the target attribute as the target first detection rules.
  • This embodiment provides a method for publishing a change event. Since the change risk prevention and control system uniformly defines attributes for describing change events, users can configure the attribute values of the attributes and the first detection rules corresponding to the attribute values to deposit the anomaly detection methods corresponding to different business systems and the anomaly detection experience of different detection personnel into the change risk prevention and control system through the first detection rules, thereby simply and quickly integrating the different anomaly detection experiences of various business teams and avoiding the situation where anomalies cannot be detected due to insufficient experience of detection personnel.
  • the multiple attributes of the change event include at least one of the following attributes: attributes representing basic information of the change event, the basic information including any of the following: title information, initiator information or time information; attributes representing type information of the change event, the type information including any of the following: code release type or parameter configuration type; attributes representing execution information of the change event, the execution information including any of the following: batch release mode information, blue-green release mode information or shutdown release mode information; attributes representing impact scope information of the change event, the impact scope information including any of the following: affected object information or impact degree information.
  • Downtime release means shutting down the business system before the change event is released, stopping user access to the business system, and then updating the applications running in the server cluster.
  • Batch release includes canary release and grayscale release.
  • Canary release means first releasing the change event to some servers in the server cluster, so that some business system users can access the upgraded business system and use real user data to verify the traffic. If the verification passes, the change event is released to the remaining servers, otherwise the code is rolled back, that is, the version before the change event is released.
  • Grayscale release is an extension of canary release, which is to divide the release into different batches, and the number of servers in each batch increases step by step. If no problems are found in the current batch after the change event is released, it will enter the next batch until the change event release of the server cluster is completed.
  • Blue-green release means that there are two completely identical and independent production environments, one called “blue environment” and the other called “green environment”.
  • the green environment is the production environment currently in use by users. Change events can be first released in the blue environment, that is, released in the server corresponding to the blue environment. Then run a smoke test in the blue environment to check whether the business system works normally after the change event is released. If the test is successful, update the routing configuration to direct the business system user traffic from the green environment to the blue environment, so that the blue environment becomes the production environment.
  • the present application provides a method for publishing a change event, and the target change event can be published in a server cluster in any of the above-mentioned publishing modes.
  • the target change event is published in the server cluster in batches, specifically including the steps shown in Figure 3:
  • Step S211 Select one or more servers to be published from the server cluster as the target servers of the current batch
  • Step S213 Publish the target change event to the target server
  • Step S22 Determine the target first detection rule from the stored rules according to the target attribute value, and perform anomaly detection on the updated application according to the target first detection rule; wherein, steps S211-S22 refer to the above embodiment and will not be repeated here.
  • Step S23 Determine whether an abnormality is detected; if so, execute step S24; if not, execute step S25.
  • Step S24 Output a notification of abnormal change and/or stop publishing target change events.
  • Step S25 Determine whether the server cluster has completed the release; if not, return to step S211, and if so, complete the release of the target change event.
  • the target change events are published in batches in the server cluster. After each batch is published, the target first detection rule is used to perform anomaly detection, and the next batch is published only after passing the detection.
  • this embodiment realizes the systematic and automatic anomaly detection of each batch of releases by the change risk prevention and control system, reducing the reliance on manual work.
  • the attributes predefined by the change risk prevention and control system for describing change events include one or more of the following: type attributes of the change event, or impact attributes of the change event, basic attributes that characterize the change event in the proposal stage, and release attributes that characterize the change event in the release stage.
  • the type attribute of the change event includes a code release type and a parameter configuration type.
  • the type attribute of the corresponding change event is a code release type.
  • the type attribute of the corresponding change event is a parameter configuration type.
  • the impact attribute of the change event includes the impact object and/or impact degree of the change event.
  • the impact object may be other applications.
  • the release of the change event of the application may not only affect the target business system to which the application belongs, but also affect the associated business systems located upstream and downstream of the target business system. Therefore, the impact object may be the associated business system.
  • the basic attributes that characterize the change event at the proposal stage refer to the attributes carried by the R&D personnel when initiating a release request for the target change event, including but not limited to one or more of the title, initiator, and initiation time of the target change event.
  • the publishing attributes characterizing the change event in the publishing stage include the publishing mode of the change event, and/or the flag information of the target server on which it is published.
  • the change event can be published in any one of the publishing modes of batch publishing, blue-green publishing, and shutdown publishing.
  • the publishing attributes include the publishing mode. If the publishing mode is the shutdown mode, the target servers are all servers in the server cluster. If the publishing mode is batch publishing or blue-green publishing, the target servers are some servers in the server cluster.
  • the flag information can be a unique flag of the server. In this way, the publishing mode of the change event and the servers on which it is published can be known through the publishing attributes.
  • the user can configure the corresponding first detection rules for different type attributes.
  • the first detection rule 1 is configured for the change event of the code release type
  • the first detection rule 2 is configured for the change event of the parameter configuration type.
  • the change event with the type attribute of the code release type is detected for anomalies according to the first detection rule 1 after being released
  • the change event with the type attribute of the parameter configuration type is detected for anomalies according to the first detection rule 2 after being released.
  • the user can configure corresponding first detection rules for different impact objects.
  • the first detection rule 3 is configured for all change events whose impact object is business system A. That is, all change events that affect business system A, including change events of applications belonging to business system A, and change events of applications of other business systems that interact with business system A, are all released for anomaly detection according to the first detection rule 3.
  • the user can configure corresponding first detection rules for different initiators of change events. For example, configure first detection rule 4 for all change events initiated by a certain initiator. After the change event is released, anomaly detection is performed according to first detection rule 4.
  • the user can configure corresponding first detection rules for different release modes. For example, configure first detection rule 5 for change events in batch release mode; configure first detection rule 6 for change events in blue-green release mode; configure first detection rule 7 for change events in shutdown release mode.
  • the method of this embodiment may also include: extending one or more extended attribute values according to the target attribute value, and searching and executing the currently matched detection rule from the stored detection rules according to the target attribute value and the one or more extended attribute values; wherein, the method for obtaining the extended attribute value includes: determining the upstream business system or downstream business system of the target business system, as well as other applications associated with the target application, based on the target application to be changed by the target change event and the target business system to which the target application belongs, and generating impact scope information including the extended attribute value of any of the upstream business system, the downstream business system or the other applications.
  • the currently matched detection rules found before the execution of the target change event include: a detection rule for detecting whether the release conditions of the target change event are met, and the release conditions include any of the following: the condition that the risk level of the target change event is less than a preset level threshold; wherein the level threshold is related to the business volume of the target business system at the current moment, and/or is related to whether the current moment is within a specified date; the target business system is the business system changed by the target change event; or, a detection rule for detecting whether the parameter value of the target parameter configured in the target change event of the parameter configuration type passes the legality check.
  • the release conditions include but are not limited to: (1) the risk level of the change event is less than a preset level threshold.
  • the level threshold is related to the business volume of the target business system at the current moment and/or whether the current moment is within a specified date.
  • the target business system is the business system described in the application.
  • the change risk prevention and control system determines the risk level threshold according to the business volume of the business system (i.e., the target business system) to which the application belongs at the current moment. For example, the larger the business volume of the target business system, the lower the level threshold. That is, when the business volume of the target business system is large, a low-risk change event can be released.
  • the business volume of the target business system exceeds a set business volume threshold, that is, during the peak business volume period, change events are not allowed to be published.
  • the risk level threshold is determined according to the business volume, and change events with risk levels below the level threshold are allowed to be published.
  • the level threshold is related to whether the current moment is within a specified date.
  • the specified date includes, but is not limited to, major national events, festivals, statutory holidays, etc.
  • the stability of the business system is particularly important, so the level threshold within the specified date can be set to be lower than the level threshold within other dates outside the specified date. That is, within the specified date, change events with a lower risk level can be released.
  • the predefined attributes of the change risk prevention and control system include the type attribute of the change event.
  • the type attribute includes the code release type and the parameter configuration type. If the parameters in the business system are modified, the type attribute of the corresponding change event is the parameter configuration type. Since the parameter values requested to be configured in the change event are manually entered by the R&D personnel, in this embodiment, the parameter values configured for the parameters to be modified (i.e., the target parameters) can be checked for legality. If the legality check is passed, it is judged that the release conditions are met.
  • a validity check may be performed based on the historical parameter values of the target parameter.
  • a value range of the target parameter may be determined. If the parameter value configured by the target change event falls within the value range, the validity check passes.
  • a change event publishing method includes the steps shown in Figure 4: in response to a publishing request for a target change event of an application, execute: Step S211: select one or more servers to be published from the server cluster as target servers for the current batch; Step S212: determine whether the publishing conditions of the change event are met; Step S213: publish the target change event to the target server; Step S22: determine the target first detection rule from the stored rules according to the target attribute value, and perform anomaly detection on the updated application according to the target first detection rule; Step S23: determine whether an anomaly is detected; if so, execute step S24; if not, execute step S25.
  • Step S24 Output a notification of abnormal change and stop issuing target change events.
  • Step S25 Determine whether the server cluster has completed the release; if not, return to step S211, and if so, complete the release of the target change event.
  • the change events are embedded and correspondingly tested before and after the release of each batch.
  • the change risk prevention and control system determines whether the current time meets the release conditions, that is, performs an accessibility test. When the release conditions are met, that is, after the accessibility test is passed, the change events of this batch are automatically released.
  • the change risk prevention and control system After the release, that is, the post-embedded point, the change risk prevention and control system performs anomaly detection. And after the anomaly detection passes, it automatically enters the accessibility test and release of the next batch.
  • this embodiment realizes automatic accessibility testing and anomaly detection by embedding points before and after each batch.
  • this embodiment in the process of batch release of change events, it can be released in a timely manner. The problem is found and the release of the change event is cut off.
  • the currently matching detection rule is found from the stored first detection rule according to the target attribute value, including: comparing the number of errors and/or error types before and after the target change event is released, and determining whether an abnormality occurs in the updated application based on the comparison result of the error reporting rule; comparing the business data transmitted between the target business system and the associated business system before and after the target change event is released, and determining whether an abnormality occurs in the updated application based on the comparison result of the business data.
  • Link rules wherein the target business system is the business system to which the application belongs, and the associated business system is the upstream and/or downstream business system of the target business system.
  • the change risk prevention and control system since the change risk prevention and control system predefines the attributes of the change event, the user can customize the corresponding first detection rules for the predefined attributes. Different change events are subject to different first detection rules. In some embodiments, different change events can also use the same second detection rule for anomaly detection, that is, the second detection rule applies to all change events. In this way, the change risk prevention and control system also stores multiple second detection rules. Then, after the target change event is released, in addition to the target first detection rule, the updated application is also detected for anomalies according to the general second detection rule.
  • the second detection rules may include but are not limited to error reporting rules and link rules.
  • the error reporting rules in the second detection rules may be used to compare the number of errors and/or error types before and after the target change event is released, and determine whether an abnormality occurs in the updated application based on the comparison result.
  • the error reporting rule indicates a comparison of the number of errors before and after the target change event is released. If the difference in the number of errors before and after the release is greater than a preset number threshold, it is determined that an abnormality has occurred.
  • the change risk prevention and control system can monitor the number of errors that occur in one or more business systems. The change risk prevention and control system can periodically count the number of errors that occur in the business system and output it in a form such as a report, a time series statistical curve, etc. In this way, it can be determined whether an abnormality has occurred in the updated application based on the increase or decrease in the number of errors before and after the release.
  • the error reporting rule indicates that the error reporting types before and after the target change event is released are compared. If a new error reporting type appears after the release, it is determined that an exception has occurred.
  • the log records various error reporting types and the number of errors corresponding to each type. Whether an exception has occurred in the updated application can be determined by identifying whether there is a new error reporting type in the log.
  • the error reporting rule indicates that the number of errors of a specified error type before and after the target change event is released is compared, and if the difference in the number of errors of the specified error type before and after the release is greater than a quantity threshold, it is determined that an exception has occurred.
  • the log records various error types and the number of errors corresponding to each type. Therefore, the difference value of the number of errors of a specified error type before and after the release can be obtained, and it is determined whether an exception has occurred.
  • the link rule in the second detection rule can be used to compare the business data transmitted between the target business system and the associated business system before and after the target change event is released, and determine whether an abnormality occurs in the updated application based on the comparison result of the business data.
  • the target business system is the business system to which the application belongs, and the associated business system is the upstream and/or downstream business system of the target business system.
  • the business system to which the application belongs is the target business system.
  • the business data transmitted between the target business system and its upstream and downstream business systems i.e., associated business systems
  • the business data transmitted between the target business system and its upstream and downstream business systems can be captured, and by comparing the differences in business data before and after the release, it can be determined whether an abnormality has occurred. For example, if the business data transmitted from the target business system to the associated business system before the release is "Success", and the business data transmitted after the release becomes "Fail", it is determined that an abnormality has occurred in the updated application.
  • the change risk prevention and control system also stores multiple second detection rules applicable to all change events. After the target change event is released, in addition to the user-defined first detection rule, the anomaly test is also performed according to the general second detection rule, eliminating the need for users corresponding to different business fields to repeatedly configure the general second detection rule.
  • the publishing function provided by the change risk prevention and control system can also be applied to the simulation trial operation stage as shown in Figure 1.
  • the target change event can be published in batches in the test servers of the pre-release environment and the test environment in the simulation trial operation stage, and can be published in batches in the server cluster of the formal environment in the publishing stage.
  • the updated application is detected for anomalies according to the stored first detection rule and the second detection rule. And after passing the anomaly detection, enter the next batch of releases.
  • FIG5B it is a schematic diagram of another change risk prevention and control system according to an exemplary embodiment of the present specification, which can be used to perform risk prevention and control on application changes of any business system among multiple other business systems.
  • the change risk prevention and control system itself can pre-integrate multiple detection capabilities, such as the defense capability integration shown in the figure, which can include multiple detection capabilities such as inspection platform, application capacity platform, fault fuse or simulation regression.
  • These defense capabilities as basic defense capabilities, can be used by the first type of users to configure detection rules to generate and store the detection rules required by the users.
  • general detection rules can also be pre-configured, such as the built-in plug-in defense capability shown in the figure, that is, the general detection rules mentioned above.
  • the figure shows the types of general rules such as change roadblocks or change conflicts as examples.
  • the change risk prevention and control system has standardized the definition of change events, and the multiple attributes of the change events include at least one of the following attributes: attributes representing basic information of the change event, the basic information including any of the following: title information, initiator information or time information; attributes representing type information of the change event, the type information including any of the following: code release type or parameter configuration type; attributes representing execution information of the change event, the execution information including any of the following: batch release mode information, blue-green release mode information or shutdown release mode information; attributes representing impact scope information of the change event, the impact scope information including any of the following: affected object information or impact degree information.
  • the change release request needs to carry the target attribute value of the attribute of the target change event for the second type of user.
  • the target attribute value configured by the second type of user can be obtained through SPI (Service provider interface).
  • SPI is a built-in standard of Java, which allows different developers to implement a specific service. Exemplarily, as shown in the figure, this embodiment can set a standard SPI model to standardize the information structure of the change event, that is, to clearly describe what information a change event should have.
  • the standard SPI model may include a change triple, a change impact surface, and a defense verification model for the attributes of the change event.
  • the change triple and the change impact surface are used to obtain the target attribute value of the change event configured by the second type of user.
  • the change triple is used to configure the following information: user representation information of the initiator of the change, target application, and type information of the change event.
  • the change triple describes which user has executed what type of change event on which application.
  • the change impact surface is an attribute that represents the impact range information of the change event, which is used for the user to configure the corresponding attribute value.
  • the defense verification model is used for the user to configure other attribute information.
  • the rule configuration module includes a first configuration submodule, which is used to implement the configurable defense function shown in the figure. Specifically, a configuration function for each of the detection capabilities can be provided, and the configuration parameters of the business system targeted by one or more of the detection capabilities are obtained by the first type of user through the configuration function and a detection rule is generated.
  • the change risk prevention and control system provides a detection capability A, which is used to monitor a certain type of business indicator to determine whether the business is Whether the business system to which the indicator belongs is abnormal.
  • the first type of user can configure a parameter for the detection capability A, which is used to describe the name K of the business system, so that the first configuration submodule can generate a detection rule of "monitoring a certain type of business of business system K to determine whether business system K is abnormal".
  • the rule configuration module of the change risk prevention and control system includes a second configuration submodule, which is used to implement the function of the code described in the figure for coded defense. Specifically, a code configuration function is provided, through which a script file describing the detection rules submitted by the second type of user is obtained, and a detection rule is generated based on the script file.
  • the detection rules of the second type of user are implemented by scripts, that is, the second type of user writes the detection rules through code, and after the system obtains the script file written by the user, it is converted into a detection rule through code analysis and stored.
  • the third configuration submodule is used to provide a service provision interface.
  • the second-category users can implement customized development through the SPI, and the detection rules submitted by the second-category users are obtained and stored through the service provision interface.
  • the second type of user is the user who needs to publish the change.
  • the second type of user initiates a change publishing request, and the change publishing request carries the target attribute value of the attribute of the target change event for the second type of user.
  • the change risk prevention and control system of this embodiment can automatically manage and control the target change event throughout the life cycle.
  • the system of this embodiment divides the change into three environments: a pre-release execution environment, a grayscale release environment, and a formal release environment.
  • the pre-launch execution environment refers to a pre-established test environment with multiple test servers for testing change events.
  • the test servers do not provide services for real users, but for virtual users. By making changes in the pre-launch execution environment, change events can be tested without disturbing real users.
  • a grayscale release environment is a release environment composed of dedicated grayscale servers that provide services to test users, who are selected from a large number of real users. By making changes in a grayscale release environment, a small number of test users can be used to test whether the changed business system is abnormal without disturbing a large number of users.
  • the official release environment refers to the release environment composed of all servers, which are used to provide services to all users. After the change event is confirmed to be normal through the pre-release execution environment and the gray release environment, the change event can be released in the official release environment.
  • this embodiment executes the change event multiple times according to the preset lifecycle management order, and the preset lifecycle management order is pre-release execution environment, gray release environment and formal release environment.
  • detection rules will be used for defense. Before execution, the detection rules are used to determine whether the change event can be executed at present; after execution, the detection rules are used to determine whether the change event will cause anomalies after execution.
  • the system can automatically find the currently matched detection rules from the stored detection rules based on the current execution environment and execute them. Based on this, the management of the entire life cycle of the change event is realized.
  • the user's judgment on the scope of impact of a change event it may be due to a variety of reasons such as the user's lack of experience, resulting in the target attribute value configured by the user may not accurately describe the scope of impact of this change event.
  • the user's change event only targets the configuration change of a parameter of a certain application in a certain business system. The user believes that the impact scope of this change event is small and only targets the changed application, so the target attribute value is configured only for this application.
  • the business system may be associated with other upstream and downstream business systems, and the application is also associated with other applications in the business system. Therefore, there is a need to detect whether other business systems and other applications are abnormal after the change.
  • the change risk prevention and control system of this embodiment can also perform an impact analysis before searching for detection rules.
  • the change execution module is used to: expand the target attribute value according to the target attribute value One or more extended attribute values are displayed, and according to the target attribute value and the one or more extended attribute values, the currently matched detection rule is searched from the stored detection rules and executed; wherein, the method for obtaining the extended attribute value includes: according to the target application to be changed by the target change event and the target business system to which the target application belongs, the upstream business system or downstream business system of the target business system, and other applications associated with the target application are determined, and the impact scope information is generated including the extended attribute value of any one of the upstream business system, the downstream business system or the other application.
  • this embodiment can expand more extended attribute values for the impact surface based on the target attribute value configured by the user, so that more suitable detection rules can be found for detection.
  • each time the currently matched detection rule is searched that is, the routing and matching of the rule shown in the figure.
  • the embodiment of this specification also provides a change risk prevention and control system 60 as shown in Figure 6.
  • the change risk prevention and control system 60 includes: a rule configuration module 610, which is used to store multiple detection rules, and the detection rules are obtained in a manner including: providing multiple predefined attributes for describing the change event, and receiving the detection rules configured by the first type of user and the attribute values of the attribute configuration applicable to the detection rules; a change publishing module 620, which is used to provide the multiple attributes and receive the change publishing request of the second type of user, and the change publishing request carries the target attribute value of the attribute of the target change event of the second type of user; a change execution module 630, which is used to execute the target change event multiple times in multiple preset execution environments according to the preset life cycle management order; wherein, before each execution of the target change event, at least according to the target attribute value, the currently matched detection rule is found from the stored detection rules and executed to determine whether the target change event
  • the multiple attributes of the change event include at least one of the following attributes: attributes representing basic information of the change event, the basic information including any of the following: title information, initiator information, or time information; attributes representing type information of the change event, the type information including any of the following: code release type or parameter configuration type; attributes representing execution information of the change event, the execution information including any of the following: batch release mode information, blue-green release mode information, or shutdown release mode information; attributes representing impact scope information of the change event, the impact scope information including any of the following: affected object information or impact degree information.
  • the change execution module is used to: expand one or more extended attribute values according to the target attribute value, and search and execute the currently matched detection rule from the stored detection rules according to the target attribute value and the one or more extended attribute values; wherein, the method for obtaining the extended attribute value includes: determining the upstream business system or downstream business system of the target business system, and other applications associated with the target application according to the target application to be changed by the target change event and the target business system to which the target application belongs, and generating the impact scope information including the extended attribute value of any of the upstream business system, the downstream business system or the other applications.
  • the change risk prevention and control system is connected to multiple business systems, and the change event is used to change one or more applications in the business system; the change risk prevention and control system integrates multiple detection capabilities; the rule configuration module includes any of the following sub-modules: a first configuration sub-module, which is used to: provide a configuration function for each of the detection capabilities, obtain the configuration parameters of the business system targeted by one or more of the detection capabilities by the second type of user through the configuration function and generate detection rules; a second configuration sub-module, which is used to: provide code configuration Function, through the code configuration function, obtain the script file describing the detection rules submitted by the second category of users, and generate detection rules based on the script file; the third configuration sub-module is used to: provide a service provision interface, obtain and store the detection rules submitted by the second category of users through the service provision interface.
  • a first configuration sub-module which is used to: provide a configuration function for each of the detection capabilities, obtain the configuration parameters of the business system targeted by one or more of the detection capabilities by the second
  • the currently matched detection rules found before the execution of the target change event include: a detection rule for detecting whether the release conditions of the target change event are met, and the release conditions include any of the following: the condition that the risk level of the target change event is less than a preset level threshold; wherein the level threshold is related to the business volume of the target business system at the current moment, and/or is related to whether the current moment is within a specified date; the target business system is the business system changed by the target change event; or, a detection rule for detecting whether the parameter value of the target parameter configured in the target change event of the parameter configuration type passes the legality check.
  • the multiple detection rules include preset detection rules; after each execution of the change event, the currently matched detection rules are found from the stored first detection rules according to the target attribute value, including: error rules that compare the number of errors and/or error types before and after the target change event is released, and determine whether an abnormality occurs in the updated application according to the comparison results; link rules that compare the business data transmitted between the target business system and the associated business systems before and after the target change event is released, and determine whether an abnormality occurs in the updated application according to the comparison results of the business data; wherein the target business system is the business system to which the application belongs, and the associated business systems are the upstream and/or downstream business systems of the target business system.
  • the multiple preset execution environments include: a pre-release execution environment, a grayscale release environment, and a formal release environment; the preset lifecycle management order is pre-release execution environment, grayscale release environment, and formal release environment.
  • the embodiment of this specification also provides a structural diagram of an electronic device as shown in Figure 7.
  • the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and of course may also include hardware required for other services.
  • the processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the method for publishing a change event described in any of the above embodiments.
  • an embodiment of this specification further provides a computer storage medium, which stores a computer program.
  • the computer program When the computer program is executed by a processor, it can be used to execute a method for publishing a change event described in any of the above embodiments.

Abstract

本说明书提供变更风险防控方法、系统、电子设备及存储介质，该系统包括：规则配置模块用于：存储多条检测规则，获取方式包括：提供预定义的用于描述变更事件的多个属性，以及接收第一类用户配置的检测规则及述检测规则适用的属性配置的属性值；变更发布模块用于：提供所述多个属性，并接收第二类用户的变更发布请求，其携带第二类用户对目标变更事件的属性的目标属性值；变更执行模块用于：在多个预设执行环境中，按预设的生命周期管理顺序多次执行所述目标变更事件；其中，在每一次执行所述目标变更事件之前及之后，至少根据目标属性值从已存储的检测规则中查找出当前匹配的检测规则并执行，以确定是否可执行该事件或确定该事件是否导致异常。

Description

变更风险防控系统、方法、电子设备及存储介质 技术领域
本说明书实施例涉及计算机技术领域，尤其涉及变更风险防控系统、方法、电子设备及存储介质。
背景技术
随着科技迭代速度的日渐提升，业务系统的升级换代也越来越频繁。业务系统包括多个应用，应用的变更事件发布会使应用更新，从而使业务系统升级。对于IT相关业务来说，内部开发人员发起的变更事件可能会对业务系统带来稳定性问题。为了维持业务系统的稳定，在变更事件发布前后可以进行异常检测。然而，不同的业务系统有不同的异常检测方法，不同的检测人员具备不同的异常检测经验。一方面，变更事件的异常检测依然依赖于人为经验，若检测人员经验不足，可能无法检查出异常。另一方面，若更换了检测人员，在先检测人员只能通过口口相授将检测经验传授给下一检测人员，未能将检测经验规则化地沉淀下来。
发明内容
本说明书实施例提供了一种变更风险防控系统、方法、电子设备及存储介质，以提供系统化的异常检测。
根据本说明书实施例的第一方面，提供一种变更风险防控系统，所述变更风险防控系统包括：规则配置模块，用于：存储多条检测规则，所述检测规则的获取方式包括：提供预定义的用于描述变更事件的多个属性，以及接收第一类用户配置的检测规则及对所述检测规则适用的属性配置的属性值；变更发布模块，用于：提供所述多个属性，并接收第二类用户的变更发布请求，所述变更发布请求携带所述第二类用户对目标变更事件的属性的目标属性值；变更执行模块，用于：在多个预设执行环境中，按预设的生命周期管理顺序多次执行所述目标变更事件；其中，在每一次执行所述目标变更事件之前，至少根据所述目标属性值从已存储的检测规则中查找出当前匹配的检测规则并执行，以确定是否可执行所述目标变更事件；以及每一次执行所述目标变更事件之后，至少根据所述目标属性值从已存储的检测规则中查找出当前匹配的检测规则并执行，以确定所述目标变更事件是否导致异常。
在一些例子中，所述变更事件的多个属性，包括如下至少一类属性：表征变更事件的基础信息的属性，所述基础信息包括如下任一：标题信息、发起方信息或时间信息；表征变更事件的类型信息的属性，类型信息包括如下任一：代码发布类型或参数配置类型；表征变更事件的执行信息的属性，所述执行信息包括如下任一：分批发布模式信息、蓝绿发布模式信息或停机发布模式信息；表征变更事件的影响范围信息的属性，所述影响范围信息包括如下任一：影响对象信息或影响程度信息。
在一些例子中，所述变更执行模块，用于：根据所述目标属性值扩展出一个或多个扩展属性值，并根据所述目标属性值和所述一个或多个扩展属性值从已存储的检测规则中查找出当前匹配的检测规则并执行；其中，所述扩展属性值的获取方式，包括：根据所述目标变更事件所要变更的目标应用以及所述目标应用所属的目标业务系统，确定所述目标业务系统的上游业务系统或下游业务系统，以及与目标应用关联的其他应用，生 成影响范围信息包括所述上游业务系统、下游业务系统或所述其他应用任一的扩展属性值。
在一些例子中，所述变更风险防控系统连接多个业务系统，所述变更事件用于对所述业务系统中一种或多种应用的变更；所述变更风险防控系统集成有多种检测能力；所述规则配置模块，包括如下任一子模块：第一配置子模块，用于：提供对每种所述检测能力的配置功能，通过所述配置功能获取所述第二类用户对一种或多个所述检测能力所针对的业务系统的配置参数并生成检测规则；第二配置子模块，用于：提供代码配置功能，通过所述代码配置功能获取第二类用户提交的描述检测规则的脚本文件，基于所述脚本文件生成检测规则；第三配置子模块，用于：提供服务提供接口，通过所述服务提供接口获取第二类用户提交的检测规则并存储。
在一些例子中，所述目标变更事件在执行之前查找出的当前匹配的检测规则，包括：用于检测是否满足目标变更事件的发布条件的检测规则，所述发布条件包括如下任一：目标变更事件的风险等级小于预设的等级阈值的条件；其中，所述等级阈值与当前时刻下目标业务系统的业务量相关，和/或与当前时刻是否在指定日期内相关；所述目标业务系统是所述目标变更事件所变更的业务系统；或，用于检测参数配置类型的目标变更事件中所配置的目标参数的参数值是否通过合法性校验的检测规则。
在一些例子中，所述多条检测规则包括预置检测规则；每一次执行所述变更事件之后，根据所述目标属性值从已存储的第一检测规则中查找出当前匹配的检测规则，包括：对比在目标变更事件发布前后的报错数量和/或报错类型，根据对比结果确定更新后的应用是否发生异常的报错规则；对比在目标变更事件发布前后目标业务系统与关联业务系统之间传递的业务数据，根据业务数据的对比结果确定更新后的应用是否发生异常的链路规则；其中，所述目标业务系统是所述应用所属的业务系统，所述关联业务系统是所述目标业务系统的上游和/或下游的业务系统。
在一些例子中，所述多个预设执行环境包括：预发执行环境、灰度发布环境和正式发布环境；所述预设的生命周期管理顺序依次为预发执行环境、灰度发布环境和正式发布环境。
根据本说明书实施例的第二方面，提供一种变更风险防控方法，所述方法包括：存储多条检测规则，所述检测规则的获取方式包括：提供预定义的用于描述变更事件的多个属性，以及接收第一类用户配置的检测规则及对所述检测规则适用的属性配置的属性值；提供所述多个属性，并接收第二类用户的变更发布请求，所述变更发布请求携带所述第二类用户对目标变更事件的属性的目标属性值；在多个预设执行环境中，按预设的生命周期管理顺序多次执行所述目标变更事件；其中，在每一次执行所述目标变更事件之前，至少根据所述目标属性值从已存储的检测规则中查找出当前匹配的检测规则并执行，以确定是否可执行所述目标变更事件；以及每一次执行所述目标变更事件之后，至少根据所述目标属性值从已存储的检测规则中查找出当前匹配的检测规则并执行，以确定所述目标变更事件是否导致异常。
根据本说明书实施例的第三方面，提供一种电子设备，所述电子设备包括：处理器；用于存储处理器可执行指令的存储器；其中，所述处理器调用所述可执行指令时实现第一方面任一所述方法的操作。
根据本说明书实施例的第四方面，提供一种计算机可读存储介质，所述计算机可读存储介质上存储有若干计算机指令，所述计算机指令被执行时执行第一方面任一所述的 方法。
本说明书实施例提供的技术方案可以包括以下有益效果：本说明书实施例提供了变更风险防控系统、方法、电子设备及存储介质，变更风险防控系统规则配置模块，用于：存储多条检测规则，所述检测规则的获取方式包括：提供预定义的用于描述变更事件的多个属性，以及接收第一类用户配置的检测规则及对所述检测规则适用的属性配置的属性值；变更发布模块，用于：提供所述多个属性，并接收第二类用户的变更发布请求，述变更发布请求携带所述第二类用户对目标变更事件的属性的目标属性值；变更执行模块，用于：在多个预设执行环境中，按预设的生命周期管理顺序多次执行所述目标变更事件；其中，在每一次执行所述目标变更事件之前，至少根据所述目标属性值从已存储的检测规则中查找出当前匹配的检测规则并执行，以确定是否可执行所述目标变更事件；以及每一次执行所述目标变更事件之后，至少根据所述目标属性值从已存储的检测规则中查找出当前匹配的检测规则并执行，以确定所述目标变更事件是否导致异常。基于此，由于变更风险防控系统提供了供第一类用户使用的规则配置模块，统一定义了用于描述变更事件的属性，因此用户可以通过配置属性的属性值、以及与属性值对应的检测规则，将各个第一类用户的异常检测经验通过检测规则沉淀至系统中，简单、快速地融合各个检测人员不同的异常检测经验，避免了因检测人员经验不足导致检测不出异常的情况。并且，对于第二类用户来说，只需要通过变更发布模块发起变更发布请求即可，变更执行模块可以在多个预设执行环境中，按预设的生命周期管理顺序多次执行所述目标变更事件，从而实现变更事件的自动管理。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本说明书实施例。
附图说明
此处的附图被并入说明书中并构成本说明书实施例的一部分，示出了符合本说明书实施例的实施例，并与说明书一起用于解释本说明书实施例的原理。
图1是本说明书根据一实施例示出的变更事件各个阶段的示意图。
图2是本说明书根据一实施例示出的一种变更风险防控方法的流程图。
图3是本说明书根据另一实施例示出的一种变更风险防控方法的流程图。
图4是本说明书根据另一实施例示出的一种变更风险防控方法的流程图。
图5A是本说明书根据另一实施例示出的一种变更风险防控方法的流程图。
图5B是本说明书根据另一实施例示出的变更风险防控系统的一种示意图。
图6是本说明书根据一实施例示出的一种变更风险防控系统的示意图。
图7是本说明书根据一实施例示出的一种电子设备的硬件结构图。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本说明书实施例相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本说明书实施例的一些方面相一致的装置和方法 的例子。
在本说明书实施例使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本说明书实施例。在本说明书实施例和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解，尽管在本说明书实施例可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本说明书实施例范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。
随着科技迭代速度的日渐提升，业务系统的升级换代也越来越频繁。以电商为例，购物车、商品详情、下单都是独立的应用，而众多应用构成了诸如购物、支付等业务系统。从技术实现的角度来说，每次业务系统的升级包括一个或多个应用的变更。换句话说，应用的变更事件发布会使应用更新，从而使业务系统升级。应用的变更包括如图1所示的5个阶段。首先，内部开发人员发起应用的变更事件请求并提交变更代码，评审人员浏览变更代码逻辑，基于漏洞异常、代码风格、性能缺陷、重复代码等维度进行变更事件的内容审批。审批通过后则进入仿真试运行，在测试环境中进行风险模拟。随后进入变更事件的发布阶段，并在发布后进行变更事件度量。
针对变更事件发起阶段与审核阶段，在业内往往通过人为管理实现。也即通过内部评审人员的审批流程，协助评估变更过程中的风险问题。但是人为管理往往因为审核的吞吐量大、变更发起人信息填写纰漏、经验不足等原因，难以对风险进行全面性地评估，使得一些潜在的风险未能在审核阶段就被发现并解决。
在此基础上，业内在保留变更事件发起与审核的同时，平台还增加了对于变更事件的感知能力。平台可以对各个变更事件进行主动的感知，主动排查变更事件可能存在的风险，使得评审人员除了可以从变更发起人提交的资料中被动地发现变更事件的风险以外，还可以主动地在评审阶段发现各种变更并发下存在的风险。但是，这样的方案依然无法在变更的发布阶段及时发现异常并作出干预。
对于IT相关业务来说，内部开发人员发起的变更事件可能会对业务系统带来稳定性问题。为了维持业务系统的稳定，在变更事件发布前后可以进行异常检测。然而，不同的业务系统有不同的异常检测方法，不同的检测人员具备不同的异常检测经验。一方面，变更事件的异常检测依然依赖于人为经验，若检测人员经验不足，可能无法检查出异常。另一方面，若更换了检测人员，在先检测人员只能通过口口相授将检测经验传授给下一检测人员，未能将检测经验规则化地沉淀下来。
相关技术中，对于变更的风险防控，大部分方案都是针对业务系统体量较少的场景下提出的，因此从变更的发布、变更事件的管理至变更事件的异常检测，都是由发起变更的技术人员所管控；而本申请方案的发明人发现，随着企业业务的增长，业务系统体量越来越多，一个企业逐渐布局越来越多的业务系统，且各个业务系统之间又相互有依赖关系，一个业务系统的技术人员只管控自己业务系统的变更，从变更事件的管理来说，各个业务系统的技术人员可能有一些工作的重复，从变更事件的异常检测来说，异常检测的规则依赖人为经验，可能由于不了解其他业务系统的情况，从而导致异常检测存在 局限性，也无法将检测经验规则化地沉淀下来以进行复用。
为此，本说明书实施例提供一种变更风险防控方法，应用于变更风险防控系统。示例性地，该变更风险防控系统可以是集开发(Development)与运维(Operations)一体化的变更风险防控系统。变更风险防控系统可以连接一个或多个服务器集群，服务器集群中每个服务器可以运行相同应用。示例性地，服务器集群中每个服务器均搭载有相同的业务系统，因此运行有相同的应用。上述方法包括如图2所示的步骤S11至步骤S13。
步骤S11：存储多条检测规则。
其中，所述检测规则的获取方式包括：提供预定义的用于描述变更事件的多个属性，以及接收第一类用户配置的检测规则及对所述检测规则适用的属性配置的属性值。
步骤S12：提供所述多个属性，并接收第二类用户的变更发布请求。
所述变更发布请求携带所述第二类用户对目标变更事件的属性的目标属性值。
步骤S13：在多个预设执行环境中，按预设的生命周期管理顺序多次执行所述目标变更事件。
其中，在每一次执行所述目标变更事件之前，至少根据所述目标属性值从已存储的检测规则中查找出当前匹配的检测规则并执行，以确定是否可执行所述目标变更事件；以及每一次执行所述目标变更事件之后，至少根据所述目标属性值从已存储的检测规则中查找出当前匹配的检测规则并执行，以确定所述目标变更事件是否导致异常。
示例性地，步骤S11与步骤S12之间没有先后执行顺序。
示例性地，变更风险防控系统可以连接多个服务器集群，不同服务器集群搭载不同的业务系统。各个业务系统之间可以存在业务数据流量的交互，也可以相互独立。各个业务系统由不同的领域专家，包括研发人员、质量人员、运维人员，进行检测规则的配置。如此，不同的领域专家可以通过变更风险防控系统提供的规则配置功能，对与本业务系统相关的变更事件配置相应的检测规则。
由于变更风险防控系统预定义有用于描述应用的变更事件的属性，不管是同一应用的不同变更事件，或是同一业务系统中不同应用的变更事件，还是不同业务系统的应用的变更事件，都采用变更风险防控系统预定义的属性进行描述。
如此，当领域专家，也即第一类用户，需要进行规则配置时，可以向变更风险防控系统发起规则配置请求。响应于该规则配置请求，变更风险防控系统会向用户发送预定义的属性。
示例性地，若预定义的属性包括多个，则向用户发送所有属性。
示例性地，变更风险防控系统可以通过标准接口向用户发送预定义的属性。例如标准接口可以是SPI(Service Provider Interface，服务提供接口)。
随后，第一类用户可以对接收到的属性配置属性值，以及配置与该属性值对应的检测规则。示例性地，用户可以对接收的所有属性中的至少部分属性配置属性值，以及配置属性值对应的至少一条第一类检测规则。本实施例将第一类用户配置的检测规则称为第一类检测规则；实际应用中，系统存储的检测规则还可以有其他的实现方式，例如，是由开发本实施例的系统的技术人员预先配置的。
作为例子，若预定义的属性包括属性1到属性10共10条属性，那么响应于规则配 置请求，变更风险防控系统可以向用户发送属性1至属性10。其中，用户可以只对其中的属性2配置属性值A，以及对属性3配置属性值B，并配置对应的至少一条第一类检测规则。例如，用户可以对属性2的属性值A配置一条或多条第一类检测规则，对属性3的属性值B配置一条或多条第一类检测规则。那么当变更事件的属性2的属性值等于A时，则命中所配置的一条或多条第一类检测规则。又例如，用户可以对属性2的属性值A，以及属性3的属性值B配置一条或多条第一类检测规则。那么当变更事件的属性2的属性值等于A，且属性3的属性值等于B时，则命中所配置的一条或多条第一类检测规则。
在用户完成配置后，变更风险防控系统接收并存储用户配置的第一类检测规则，以及对第一类检测规则使用的属性配置的属性值。
在一些例子中，通过第一类用户获取检测规则的获取方式可以包括如下任一：提供对每种所述检测能力的配置功能，通过所述配置功能获取所述第二类用户对一种或多个所述检测能力所针对的业务系统的配置参数并生成检测规则；提供代码配置功能，通过所述代码配置功能获取第二类用户提交的描述检测规则的脚本文件，基于所述脚本文件生成检测规则；提供服务提供接口，通过所述服务提供接口获取第二类用户提交的检测规则并存储。
示例性地，用户可以将配置的属性值以及对应的第一类检测规则以插件的形式上传至变更风险防控系统。
示例性地，变更风险防控系统可以向用户提供配置界面，配置界面可以展示有预定义的所有属性。此外，配置界面还可以包括配置模块，以供用户在配置模块中配置属性值以及第一检测规则。例如，配置模块可以包括属性配置子模块和规则配置子模块。用户可以在属性配置子模块中配置至少部分属性的属性值，在规则配置子模块中配置第一类检测规则。
示例性地，变更风险防控系统存储第一类检测规则、以及第一类检测规则与属性值的映射关系。作为例子，可以以表达式的形式存储第一检测规则与属性值的映射关系。表达式可以是等式，等式的一侧可以记录用户配置的属性的属性值，另一侧可以记录第一类检测规则或第一类检测规则的唯一标识。利用唯一标识，可以读取对应的第一类检测规则。
此外，变更风险防控系统还提供变更事件发布功能，研发人员可以使用变更风险防控系统提供的变更事件发布功能进行应用的变更事件发布。其中，变更风险防控系统的规则配置功能与变更事件发布功能可以由同一用户调用，也可以由不同用户调用。
本实施例中，还可以供第二类用户发布针对变更事件的变更发布请求，本实施例将第二类用户发布的一次变更事件称为目标变更事件。本实施例的第二类用户包括具有变更需求的一类用户。由于变更风险防控系统预定义有用于描述应用的变更事件的属性，因此，当研发人员向变更风险防控系统发起应用的目标变更事件的发布请求时，发布请求携带目标变更事件的属性的目标属性值。可选地，若预定义的属性包括多个，则至少部分属性的目标属性值由研发人员配置，其余属性的目标属性值由变更风险防控系统生成。
本实施例方案中，可以在多个预设执行环境中，按预设的生命周期管理顺序多次执行所述目标变更事件；其中，在每一次执行所述目标变更事件之前，至少根据所述目标属性值从已存储的检测规则中查找出当前匹配的检测规则并执行，以确定是否可执行所 述目标变更事件；以及每一次执行所述目标变更事件之后，至少根据所述目标属性值从已存储的检测规则中查找出当前匹配的检测规则并执行，以确定所述目标变更事件是否导致异常。
示例性地，所述多个预设执行环境包括：预发执行环境、灰度发布环境和正式发布环境；所述预设的生命周期管理顺序依次为预发执行环境、灰度发布环境和正式发布环境。
其中，预发执行环境指的在预先建立好的测试环境，测试环境中有多台测试服务器用于对变更事件进行测试，测试服务器不为真正的用户提供服务，而是为虚拟出来的用户提供服务。通过在预发执行环境中进行变更，能够测试变更事件而不对真实用户造成打扰。
灰度发布环境指的是由专门的灰度服务器组成的发布环境，这些灰度服务器为测试用户提供服务，测试用户是在大量的真实用户中选择得到的用户。通过在灰度发布环境进行变更，能够通过少量的测试用户测试变更后的业务系统是否异常，而不对大量用户造成打扰。
正式发布环境指的是由全部的服务器所组成的发布环境，这些服务器用于为全部用户提供服务。当通过预发执行环境和灰度发布环境确定变更事件无异常后，即可以在正式发布环境中发布变更事件。
作为例子，以灰度发布环境或正式发布环境为例，响应于上述发布请求，变更风险防控系统可以将目标变更事件发布至服务器集群的目标服务器中。所谓目标变更事件的发布，可以理解为更新目标服务器中所运行的目标变更事件对应的应用。在发布以后，可以根据发布请求携带的属性的目标属性值，从存储的规则中确定目标第一检测规则，并根据目标第一检测规则对更新后的应用进行异常检测。其中，目标第一检测规则可以是用户按照上述方法历史配置并存储至变更风险防控系统的第一检测规则。
示例性地，可以根据目标属性值，以及上述变更风险防控系统存储的第一检测规则与属性值的映射关系，确定出目标第一检测规则，每次匹配到的检测可以有一条或多条。
示例性地，可以遍历变更风险防控系统中所有的表达式，确定命中目标属性的所有第一类检测规则作为目标第一检测规则。
本实施例提供的一种变更事件的发布方法，由于变更风险防控系统统一定义了用于描述变更事件的属性，因此用户可以通过配置属性的属性值、以及与属性值对应的第一检测规则，将不同业务系统对应的异常检测方法、以及不同检测人员的异常检测经验，通过第一检测规则沉淀至变更风险防控系统中，简单、快速地融合各个业务团队不同的异常检测经验，避免了因检测人员经验不足导致检测不出异常的情况。
所述变更事件的多个属性，包括如下至少一类属性：表征变更事件的基础信息的属性，所述基础信息包括如下任一：标题信息、发起方信息或时间信息；表征变更事件的类型信息的属性，类型信息包括如下任一：代码发布类型或参数配置类型；表征变更事件的执行信息的属性，所述执行信息包括如下任一：分批发布模式信息、蓝绿发布模式信息或停机发布模式信息；表征变更事件的影响范围信息的属性，所述影响范围信息包括如下任一：影响对象信息或影响程度信息。
示例性的，变更事件的发布模式包括多种，例如包括但不限于停机发布、分批发布、蓝绿发布等。
停机发布，即在变更事件发布以前关闭业务系统，停止业务系统的用户访问，然后更新服务器集群中运行的应用。分批发布包括金丝雀发布与灰度发布。金丝雀发布是指在先将变更事件发布至服务器集群的部分服务器中，使得部分业务系统用户访问升级后的业务系统，用真实的用户数据来做流量验证。若验证通过则变更事件发布至剩余的服务器，否则回滚代码，即恢复变更事件发布前的版本。灰度发布是金丝雀发布的延伸，是将发布分成不同批次，每批次的服务器数量逐级增加。若变更事件发布后在当前批次中没有发现问题，则进入下一批次，直到完成服务器集群的变更事件发布。
蓝绿发布，是指有两个完全相同、相互独立的生产环境，一个叫“蓝环境”，一个叫“绿环境”。其中，绿环境是用户正在使用的生产环境。变更事件可以先发布在蓝环境中，也即发布在蓝环境对应的服务器中。然后在蓝环境中运行冒烟测试，以检查变更事件发布后的业务系统是否正常工作。若测试通则，则更新路由配置，将业务系统用户流量从绿环境导向蓝环境，使得蓝环境变成生产环境。
本申请提供的一种变更事件的发布方式，目标变更事件可以以上述任一发布模式发布在服务器集群中。示例性地，目标变更事件分批发布在服务器集群中，具体包括如图3所示的步骤：响应于对应用的目标变更事件的发布请求，执行：步骤S211：从所述服务器集群中选取一个或多个待发布的服务器作为当前批次的目标服务器；步骤S213：将所述目标变更事件发布至所述目标服务器；步骤S22：根据所述目标属性值从存储的规则中确定目标第一检测规则，并根据所述目标第一检测规则对更新后的应用进行异常检测；其中，步骤S211-步骤S22参见上文实施例，在此不再赘述。
步骤S23：判断是否检测出异常；若是，执行步骤S24；若否，执行步骤S25。
步骤S24：输出变更异常的通知和/或停止目标变更事件的发布。
步骤S25：判断服务器集群是否完成发布；若否，返回执行步骤S211，若是，完成目标变更事件的发布。
在本实施例中，目标变更事件分批地发布在服务器集群中，在每一批次的发布后均使用目标第一检测规则进行异常检测，且在通过检测后才进行下一批次的发布。与相关技术中人工确认的检查逻辑相比，本实施例实现了由变更风险防控系统系统化且自动化地对每一批次的发布进行异常检测，减少了对人工的依赖。
在一些实施例中，变更风险防控系统预定义的用于描述变更事件的属性包括以下一种或多种：变更事件的类型属性、或变更事件的影响属性、表征变更事件在提出阶段的基础属性、表征变更事件在发布阶段的发布属性。
示例性地，变更事件的类型属性包括代码发布类型和参数配置类型。例如，若增加和/或修改业务系统提供的功能，则相应的变更事件的类型属性为代码发布类型。又例如，若对业务系统中的参数进行修改，则相应的变更事件的类型属性为参数配置类型。
示例性地，变更事件的影响属性包括变更事件的影响对象和/或影响程度。例如，影响对象可以是其他应用。又例如，应用的变更事件发布除了会影响到该应用所属的目标业务系统，还可能影响到位于目标业务系统上下游的关联业务系统。因此，影响对象可以是关联业务系统。
示例性地，表征变更事件在提出阶段的基础属性，是指研发人员在发起目标变更事件的发布请求时所携带的属性，包括但不限于目标变更事件的标题、发起方、发起时间中的一种或多种。
示例性地，表征变更事件在发布阶段的发布属性包括变更事件的发布模式，和/或发布在的目标服务器的标志信息。如上所述，变更事件可以以分批发布、蓝绿发布、和停机发布中的任一种发布模式进行发布。如此，发布属性包括发布模式。若发布模式为停机模式，则目标服务器为服务器集群中的所有服务器。若发布模式为分批发布或蓝绿发布，则目标服务器为服务器集群中的部分服务器。标志信息可以是服务器的唯一标志。如此，通过发布属性可以得知变更事件的发布模式，以及在哪些服务器中发布。
作为例子，用户可以针对不同的类型属性配置相应的第一检测规则。例如，针对代码发布类型的变更事件配置第一检测规则①，针对参数配置类型的变更事件配置第一检测规则②。如此，类型属性为代码发布类型的变更事件在发布后依据第一检测规则①进行异常检测；类型属性为参数配置类型的变更事件在发布后依据第一检测规则②进行异常检测。
作为例子，用户可以针对不同的影响对象配置相应的第一检测规则。例如，针对影响对象为业务系统A的所有变更事件配置第一检测规则③。也即所有会影响到的业务系统A的变更事件，包括属于业务系统A的应用的变更事件，以及与业务系统A有业务数据交互的其他业务系统的应用的变更事件，在发布均依据第一检测规则③进行异常检测。
作为例子，用户可以针对不同的变更事件的发起方配置相应的第一检测规则。例如，针对某一发起方所发起的所有变更事件配置第一检测规则④。在变更事件发布后依据第一检测规则④进行异常检测。
作为例子，用户可以针对不同的发布模式配置相应的第一检测规则。例如，针对分批发布模式的变更事件配置第一检测规则⑤；针对蓝绿发布模式的变更事件配置第一检测规则⑥；针对停机发布模式的变更事件配置第一检测规则⑦。
在一些例子中，本实施例方法还可以包括：根据所述目标属性值扩展出一个或多个扩展属性值，并根据所述目标属性值和所述一个或多个扩展属性值从已存储的检测规则中查找出当前匹配的检测规则并执行；其中，所述扩展属性值的获取方式，包括：根据所述目标变更事件所要变更的目标应用以及所述目标应用所属的目标业务系统，确定所述目标业务系统的上游业务系统或下游业务系统，以及与目标应用关联的其他应用，生成影响范围信息包括所述上游业务系统、下游业务系统或所述其他应用任一的扩展属性值。
在一些例子中，所述目标变更事件在执行之前查找出的当前匹配的检测规则，包括：用于检测是否满足目标变更事件的发布条件的检测规则，所述发布条件包括如下任一：目标变更事件的风险等级小于预设的等级阈值的条件；其中，所述等级阈值与当前时刻下目标业务系统的业务量相关，和/或与当前时刻是否在指定日期内相关；所述目标业务系统是所述目标变更事件所变更的业务系统；或，用于检测参数配置类型的目标变更事件中所配置的目标参数的参数值是否通过合法性校验的检测规则。
在一些实施例中，响应于发布请求，在将目标变更事件发布到目标服务器前，还包括：判断是否满足变更事件的发布条件。
其中，发布条件包括但不限于：(1)变更事件的风险等级小于预设的等级阈值。其中，等级阈值与当前时刻下目标业务系统的业务量相关，和/或与当前时刻是否在指定日期内相关。目标业务系统是应用所述的业务系统。
变更事件的风险等级越高，代表变更事件可能带来的稳定性问题概率越大。
示例性地，针对某一应用的变更事件的发布，变更风险防控系统根据当前时刻下该应用所属业务系统(即目标业务系统)的业务量，确定风险的等级阈值。例如，目标业务系统的业务量越大，等级阈值越低。也即在目标业务系统的业务量较大时，低风险的变更事件可以发布。
可选地，当目标业务系统的业务量超过设定的业务量阈值时，也即在业务量高峰期时，不允许变更事件发布。当目标系统的业务量未超过上述业务量阈值时，根据业务量确定风险的等级阈值，并允许风险等级在等级阈值以下的变更事件发布。
示例性地，等级阈值与当前时刻是否在指定日期内相关。作为例子，指定日期包括但不限于国家重大活动、节日、法定假期等。在上述的指定日期内，业务系统的稳定显得尤其重要，因此可以设定指定日期内的等级阈值小于指定日期以外的其他日期内的等级阈值。也即在指定日期内，风险等级较小的变更事件可以发布。
(2)若目标变更事件的类型属性为参数配置类型，所配置的目标参数的参数值通过合法性校验。
如上所述，变更风险防控系统预定义的属性包括变更事件的类型属性。而类型属性则包括代码发布类型和参数配置类型。若对业务系统中的参数进行修改，则相应的变更事件的类型属性为参数配置类型。由于变更事件中请求配置的参数值由研发人员手动输入，因此在本实施例中，可以对待修改的参数(即目标参数)所配置的参数值进行合法性校验。若通过合法性校验，则判断满足发布条件。
可选地，可以根据目标参数的历史参数值进行合法性校验。作为例子，根据目标参数的历史参数值，可以确定目标参数的取值范围。若目标变更事件所配置的参数值落入该取值范围内，则通过合法性校验。
在本实施例中，若目标变更事件的发布模式为分批发布，本实施例提供的一种变更事件的发布方法包括如图4所示的步骤：响应于对应用的目标变更事件的发布请求，执行：步骤S211：从所述服务器集群中选取一个或多个待发布的服务器作为当前批次的目标服务器；步骤S212：判断是否满足变更事件的发布条件；步骤S213：将所述目标变更事件发布至所述目标服务器；步骤S22：根据所述目标属性值从存储的规则中确定目标第一检测规则，并根据所述目标第一检测规则对更新后的应用进行异常检测；步骤S23：判断是否检测出异常；若是，执行步骤S24；若否，执行步骤S25。
步骤S24：输出变更异常的通知，并停止目标变更事件的发布。
步骤S25：判断服务器集群是否完成发布；若否，返回执行步骤S211，若是，完成目标变更事件的发布。
上述步骤参见上文实施例，在此不再赘述。
在图4所示的实施例中，变更事件在每一批次的发布前后均埋点进行相应的测试。在发布前，也即前置埋点，变更风险防控系统判断当前时间是否满足发布条件，也即进行准入性测试。在满足发布条件的情况下，也即准入性测试通过后，自动发布该批次的变更事件。在发布后，也即后置埋点，变更风险防控系统进行异常检测。且在异常检测通过后自动进入下一批次的准入性测试与发布。一方面，与相关技术中人工判断目标变更事件的发布时机与人工进行异常检测相比，本实施例通过在每批次前后埋点实现了自动进行准入性测试与异常检测。另一方面，在变更事件分批发布的过程中，可以及时发 现问题，并熔断变更事件的发布。
在一些例子中，每一次执行所述变更事件之后，根据所述目标属性值从已存储的第一检测规则中查找出当前匹配的检测规则，包括：对比在目标变更事件发布前后的报错数量和/或报错类型，根据对比结果确定更新后的应用是否发生异常的报错规则；对比在目标变更事件发布前后目标业务系统与关联业务系统之间传递的业务数据，根据业务数据的对比结果确定更新后的应用是否发生异常的链路规则；其中，所述目标业务系统是所述应用所属的业务系统，所述关联业务系统是所述目标业务系统的上游和/或下游的业务系统。
在上述实施例中，由于变更风险防控系统预定义了变更事件的属性，因此用户可以针对预定义的属性自定义地配置相应的第一检测规则。不同变更事件适用不同的第一检测规则。在一些实施例中，不同的变更事件还可以使用相同的第二检测规则进行异常检测，也即第二检测规则适用于所有变更事件。如此，变更风险防控系统还存储有多条第二检测规则，那么在目标变更事件发布后，除了根据目标第一检测规则，还根据通用的第二检测规则对更新后的应用进行异常检测。其中，第二检测规则可以包括但不限于报错规则和链路规则。
示例性地，可以根据第二检测规则中的报错规则，对比在目标变更事件发布前后的报错数量和/或报错类型，根据对比结果确定更新后的应用是否发生异常。
可选地，报错规则指示对比在目标变更事件发布前后的报错数量，若发布前后报错数量的差异大于预设的数量阈值，确定发生异常。作为例子，变更风险防控系统可以监控一个或多个业务系统发生的报错数量。变更风险防控系统可以周期性地统计业务系统发生的报错数量，并以诸如报表、时序统计曲线等形式输出。如此，根据发布前后报错数量的跌涨可以判断更新后的应用是否发生异常。
可选地，报错规则指示对比在目标变更事件发布前后的报错类型，若发布后出现新的报错类型，确定发生异常。作为例子，日志中记载有各种报错类型以及每种类型对应的报错数量。可以通过识别日志中是否有新增的报错类型来判断更新后的应用是否发生异常。
可选地，报错规则指示对比在目标变更事件发布前后的指定报错类型的报错数量，若发布前后指定报错类型的报错数量的差异大于数量阈值，确定发生异常。作为例子，如上所述，日志中记载有各种报错类型以及每种类型对应的报错数量。因此可以获取指定报错类型的报错数量在发布前后的差异值，并确定是否发生异常。
示例性地，可以根据第二检测规则中的链路规则，对比在目标变更事件发布前后目标业务系统与关联业务系统之间传递的业务数据，根据业务数据的对比结果确定更新后的应用是否发生异常。其中，目标业务系统是应用所属的业务系统，关联业务系统是目标业务系统的上游和/或下游的业务系统。
针对某一应用的变更事件的发布，该应用所属业务系统即为目标业务系统。如上所述，业务系统之间可能存在业务数据流量的交互。因此，可以抓取目标业务系统与其上下游的业务系统(即关联业务系统)之间传递的业务数据，并通过对比发布前后业务数据的差异判断是否发生异常。例如，若发布前目标业务系统向关联业务系统传递的业务数据为“Success”，而发布后传递的业务数据变为“Fail”，则判断更新后的应用发生异常。
在本实施例中，变更风险防控系统还存储有多条适用于所有变更事件的第二检测规则。目标变更事件在发布后除了根据用户自定义的第一检测规则，还根据通用的第二检测规则进行异常测试，免去了不同业务领域对应的用户重复配置通用的第二检测规则。
在一些实施例中，变更风险防控系统提供的发布功能还可以应用在如图1所示的仿真试运行阶段。如图5A所示，也即在仿真试运行阶段以及发布阶段中，目标变更事件在仿真试运行阶段中可以分批次地发布在预发环境和测试环境的测试服务器中，在发布阶段中可以分批次地发布在正式环境的服务器集群中。在每一批次的发布前判断是否满足上文记载的发布条件。在每一次批次的发布后根据存储的第一检测规则与第二检测规则对更新后的应用进行异常检测。且在通过异常检测后再进入下一批次的发布。
如图5B所示，是本说明书根据一示例性实施例示出的另一种变更风险防控系统的示意图，能用于对其他多个业务系统中任一业务系统的应用变更进行风险防控。
示例性的，变更风险防控系统本身可以预先集成多种检测能力，例如图中示出的防御能力集成，可以包括巡检平台、应用容量平台、故障熔断或仿真回归等多种检测能力，这些防御能力作为基础的防御能力，可以供第一类用户进行检测规则的配置，以生成用户所需的检测规则并存储。
变更风险防控系统中，也可以预先配置通用的检测规则，例如图中示出的内置插件防御能力，即前述的通用的检测规则，图中作为示例，示出了通用规则的类型如变更路障或变更冲突等。
示例性的，变更风险防控系统对变更事件进行了标准化的定义，所述变更事件的多个属性，包括如下至少一类属性：表征变更事件的基础信息的属性，所述基础信息包括如下任一：标题信息、发起方信息或时间信息；表征变更事件的类型信息的属性，类型信息包括如下任一：代码发布类型或参数配置类型；表征变更事件的执行信息的属性，所述执行信息包括如下任一：分批发布模式信息、蓝绿发布模式信息或停机发布模式信息；表征变更事件的影响范围信息的属性，所述影响范围信息包括如下任一：影响对象信息或影响程度信息。
第二类用户提交目标变更事件时，变更发布请求需要携带所述第二类用户对目标变更事件的属性的目标属性值。其中，本实施例可以通过SPI(服务供给接口，Service provider interface)获取第二类用户配置的目标属性值。SPI是Java的一个内置标准，允许不同的开发者去实现某个特定的服务。示例性的，如图所示，本实施例可以设定一标准SPI模型，用于对变更事件的信息结构进行标准化定义，即描述清楚一个变更事件应该具备哪些信息。该标准SPI模型可以包括用于对变更事件的属性进行变更三元组、变更影响面和防御校验模型。其中，变更三元组和变更影响面用于获取第二类用户配置的本次变更事件的目标属性值。其中，变更三元组用于供配置如下信息：发起变更的用户表示信息，目标应用，变更事件的类型信息，该变更三元组即描述了哪个用户对哪个应用，执行了什么类型的变更事件。变更影响面即表征变更事件的影响范围信息的属性，用于供用户配置对应的属性值。防御校验模型用于供用户配置其他属性信息。
规则配置模块包括第一配置子模块，用于实现图中所示的配置化的防御的功能。具体的，可以提供对每种所述检测能力的配置功能，通过所述配置功能获取所述第一类用户对一种或多个所述检测能力所针对的业务系统的配置参数并生成检测规则。例如，变更风险防控系统提供一检测能力A，用于对某类业务指标进行监控，以判断该类业务 指标所属的业务系统是否异常。第一类用户可以对该检测能力A配置一参数，该参数用于描述业务系统的名称K，从而该第一配置子模块可以生成“对业务系统K的某类业务进行监控，以判断业务系统K是否异常”的检测规则。
变更风险防控系统的规则配置模块包括第二配置子模块，用于实现图中所述的代码用于代码化的防御的功能。具体的，提供代码配置功能，通过所述代码配置功能获取第二类用户提交的描述检测规则的脚本文件，基于所述脚本文件生成检测规则。本实施例中，第二类用户的检测规则采用脚本实现，即第二类用户通过代码写好检测规则，系统获取用户写好的脚本文件后，通过代码分析转换为检测规则并存储。
第三配置子模块，用于：提供服务提供接口，本实施例通过SPI可以供各个第二类用户实现定制化开发，通过所述服务提供接口获取第二类用户提交的检测规则并存储。
如图所示，第二类用户是需要发布变更的用户，第二类用户发起变更发布请求，变更发布请求携带所述第二类用户对目标变更事件的属性的目标属性值。本实施例的变更风险防控系统能够对目标变更事件进行整个生命周期的自动管控。本实施例的系统将变更拆分为三个环境：预发执行环境、灰度发布环境和正式发布环境。
其中，预发执行环境指的在预先建立好的测试环境，测试环境中有多台测试服务器用于对变更事件进行测试，测试服务器不为真正的用户提供服务，而是为虚拟出来的用户提供服务。通过在预发执行环境中进行变更，能够测试变更事件而不对真实用户造成打扰。
灰度发布环境指的是由专门的灰度服务器组成的发布环境，这些灰度服务器为测试用户提供服务，测试用户是在大量的真实用户中选择得到的用户。通过在灰度发布环境进行变更，能够通过少量的测试用户测试变更后的业务系统是否异常，而不对大量用户造成打扰。
正式发布环境指的是由全部的服务器所组成的发布环境，这些服务器用于为全部用户提供服务。当通过预发执行环境和灰度发布环境确定变更事件无异常后，即可以在正式发布环境中发布变更事件。
如图所示，本实施例按预设的生命周期管理顺序多次执行所述变更事件，预设的生命周期管理顺序依次为预发执行环境、灰度发布环境和正式发布环境。并且，每一次变更事件的执行之前(pre)以及之后(post)，均会利用检测规则进行防御。在执行之前，通过检测规则确定变更事件当前是否可以执行；在执行之后，通过检测规则确定变更事件执行后是否会导致异常。如图中的每一个pre或post的时间点，系统均可以自动基于当前所处的执行环境，从已存储的检测规则中查找出当前匹配的检测规则并执行。基于此，实现了对变更事件的整个生命周期的管理。
在一些例子中，考虑到用户对一次变更事件的影响范围的判断，可能基于其自身经验的不足等多种原因，导致其配置的目标属性值可能无法准确地描述本次变更事件的影响范围。例如，用户的本次变更事件仅针对某个业务系统中某个应用的一个参数的配置变更，用户认为本次变更事件的影响范围较小，仅针对该被变更的应用，因此目标属性值的配置仅针对该应用。然而，实际应用中该业务系统可能关联上下游的其他业务系统，该应用也关联业务系统中的其他应用，因此，对其他业务系统和其他应用均有检测变更后是否异常的需要。基于此，本实施例的变更风险防控系统在查找检测规则之前，还可以进行影响面分析。示例性的，所述变更执行模块，用于：根据所述目标属性值扩 展出一个或多个扩展属性值，并根据所述目标属性值和所述一个或多个扩展属性值从已存储的检测规则中查找出当前匹配的检测规则并执行；其中，所述扩展属性值的获取方式，包括：根据所述目标变更事件所要变更的目标应用以及所述目标应用所属的目标业务系统，确定所述目标业务系统的上游业务系统或下游业务系统，以及与目标应用关联的其他应用，生成影响范围信息包括所述上游业务系统、下游业务系统或所述其他应用任一的扩展属性值。
也即是，本实施例可以在用户配置的目标属性值的基础上，针对影响面扩展出更多的扩展属性值，从而可以查找出更多合适的检测规则进行检测。
在确定了目标属性值和扩展属性值后，每一次查找当前匹配的检测规则，即图中示出的规则的路由和匹配。查找的检测规则可能有很多，在执行时，可以如图中所示，进行规则的并发调用。
基于上述任意实施例所述的一种变更事件的发布方法，本说明书实施例还提供了如图6所示的一种变更风险防控系统60。如图6所示，变更风险防控系统60包括：规则配置模块610，用于：存储多条检测规则，所述检测规则的获取方式包括：提供预定义的用于描述变更事件的多个属性，以及接收第一类用户配置的检测规则及对所述检测规则适用的属性配置的属性值；变更发布模块620，用于：提供所述多个属性，并接收第二类用户的变更发布请求，所述变更发布请求携带所述第二类用户对目标变更事件的属性的目标属性值；变更执行模块630，用于：在多个预设执行环境中，按预设的生命周期管理顺序多次执行所述目标变更事件；其中，在每一次执行所述目标变更事件之前，至少根据所述目标属性值从已存储的检测规则中查找出当前匹配的检测规则并执行，以确定是否可执行所述目标变更事件；以及每一次执行所述目标变更事件之后，至少根据所述目标属性值从已存储的检测规则中查找出当前匹配的检测规则并执行，以确定所述目标变更事件是否导致异常。
在一些例子中，所述变更事件的多个属性，包括如下至少一类属性：表征变更事件的基础信息的属性，所述基础信息包括如下任一：标题信息、发起方信息或时间信息；表征变更事件的类型信息的属性，类型信息包括如下任一：代码发布类型或参数配置类型；表征变更事件的执行信息的属性，所述执行信息包括如下任一：分批发布模式信息、蓝绿发布模式信息或停机发布模式信息；表征变更事件的影响范围信息的属性，所述影响范围信息包括如下任一：影响对象信息或影响程度信息。
在一些例子中，所述变更执行模块，用于：根据所述目标属性值扩展出一个或多个扩展属性值，并根据所述目标属性值和所述一个或多个扩展属性值从已存储的检测规则中查找出当前匹配的检测规则并执行；其中，所述扩展属性值的获取方式，包括：根据所述目标变更事件所要变更的目标应用以及所述目标应用所属的目标业务系统，确定所述目标业务系统的上游业务系统或下游业务系统，以及与目标应用关联的其他应用，生成影响范围信息包括所述上游业务系统、下游业务系统或所述其他应用任一的扩展属性值。
在一些例子中，所述变更风险防控系统连接多个业务系统，所述变更事件用于对所述业务系统中一种或多种应用的变更；所述变更风险防控系统集成有多种检测能力；所述规则配置模块，包括如下任一子模块：第一配置子模块，用于：提供对每种所述检测能力的配置功能，通过所述配置功能获取所述第二类用户对一种或多个所述检测能力所针对的业务系统的配置参数并生成检测规则；第二配置子模块，用于：提供代码配置 功能，通过所述代码配置功能获取第二类用户提交的描述检测规则的脚本文件，基于所述脚本文件生成检测规则；第三配置子模块，用于：提供服务提供接口，通过所述服务提供接口获取第二类用户提交的检测规则并存储。
在一些例子中，所述目标变更事件在执行之前查找出的当前匹配的检测规则，包括：用于检测是否满足目标变更事件的发布条件的检测规则，所述发布条件包括如下任一：目标变更事件的风险等级小于预设的等级阈值的条件；其中，所述等级阈值与当前时刻下目标业务系统的业务量相关，和/或与当前时刻是否在指定日期内相关；所述目标业务系统是所述目标变更事件所变更的业务系统；或，用于检测参数配置类型的目标变更事件中所配置的目标参数的参数值是否通过合法性校验的检测规则。
在一些例子中，所述多条检测规则包括预置检测规则；每一次执行所述变更事件之后，根据所述目标属性值从已存储的第一检测规则中查找出当前匹配的检测规则，包括：对比在目标变更事件发布前后的报错数量和/或报错类型，根据对比结果确定更新后的应用是否发生异常的报错规则；对比在目标变更事件发布前后目标业务系统与关联业务系统之间传递的业务数据，根据业务数据的对比结果确定更新后的应用是否发生异常的链路规则；其中，所述目标业务系统是所述应用所属的业务系统，所述关联业务系统是所述目标业务系统的上游和/或下游的业务系统。
在一些例子中，所述多个预设执行环境包括：预发执行环境、灰度发布环境和正式发布环境；所述预设的生命周期管理顺序依次为预发执行环境、灰度发布环境和正式发布环境。
基于上述任意实施例所述的一种变更事件的发布方法，本说明书实施例还提供了如图7所示的一种电子设备的结构示意图。如图7，在硬件层面，该电子设备包括处理器、内部总线、网络接口、内存以及非易失性存储器，当然还可能包括其他业务所需要的硬件。处理器从非易失性存储器中读取对应的计算机程序到内存中然后运行，以实现上述任意实施例所述的一种变更事件的发布方法。
基于上述任意实施例所述的一种变更事件的发布方法，本说明书实施例还提供了一种计算机存储介质，存储介质存储有计算机程序，计算机程序被处理器执行时可用于执行上述任意实施例所述的一种变更事件的发布方法。
上述对本说明书实施例特定实施例进行了描述。其它实施例在所附权利要求书的范围内。在一些情况下，在权利要求书中记载的动作或步骤可以按照不同于实施例中的顺序来执行并且仍然可以实现期望的结果。另外，在附图中描绘的过程不一定要求示出的特定顺序或者连续顺序才能实现期望的结果。在某些实施方式中，多任务处理和并行处理也是可以的或者可能是有利的。
本领域技术人员在考虑说明书及实践这里申请的发明后，将容易想到本说明书实施例的其它实施方案。本说明书实施例旨在涵盖本说明书实施例的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本说明书实施例的一般性原理并包括本说明书实施例未申请的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本说明书实施例的真正范围和精神由下面的权利要求指出。

Claims (10)

  1. 一种变更风险防控系统，所述变更风险防控系统包括：
    规则配置模块，用于：存储多条检测规则，所述检测规则的获取方式包括：提供预定义的用于描述变更事件的多个属性，以及接收第一类用户配置的检测规则及对所述检测规则适用的属性配置的属性值；
    变更发布模块，用于：提供所述多个属性，并接收第二类用户的变更发布请求，所述变更发布请求携带所述第二类用户对目标变更事件的属性的目标属性值；
    变更执行模块，用于：在多个预设执行环境中，按预设的生命周期管理顺序多次执行所述目标变更事件；其中，在每一次执行所述目标变更事件之前，至少根据所述目标属性值从已存储的检测规则中查找出当前匹配的检测规则并执行，以确定是否可执行所述目标变更事件；以及每一次执行所述目标变更事件之后，至少根据所述目标属性值从已存储的检测规则中查找出当前匹配的检测规则并执行，以确定所述目标变更事件是否导致异常。
  2. 根据权利要求1所述的系统，所述变更事件的多个属性，包括如下至少一类属性：
    表征变更事件的基础信息的属性，所述基础信息包括如下任一：标题信息、发起方信息或时间信息；
    表征变更事件的类型信息的属性，类型信息包括如下任一：代码发布类型或参数配置类型；
    表征变更事件的执行信息的属性，所述执行信息包括如下任一：分批发布模式信息、蓝绿发布模式信息或停机发布模式信息；
    表征变更事件的影响范围信息的属性，所述影响范围信息包括如下任一：影响对象信息或影响程度信息。
  3. 根据权利要求2所述的系统，所述变更执行模块，用于：
    根据所述目标属性值扩展出一个或多个扩展属性值，并根据所述目标属性值和所述一个或多个扩展属性值从已存储的检测规则中查找出当前匹配的检测规则并执行；其中，所述扩展属性值的获取方式，包括：
    根据所述目标变更事件所要变更的目标应用以及所述目标应用所属的目标业务系统，确定所述目标业务系统的上游业务系统和/或下游业务系统，以及与目标应用关联的其他应用，生成影响范围信息包括所述上游业务系统、下游业务系统或所述其他应用任一的扩展属性值。
  4. 根据权利要求1所述的系统，所述变更风险防控系统连接多个业务系统，所述变更事件用于对所述业务系统中一种或多种应用的变更；所述变更风险防控系统集成有多种检测能力；
    所述规则配置模块，包括如下任一子模块：
    第一配置子模块，用于：提供对每种所述检测能力的配置功能，通过所述配置功能获取所述第二类用户对一种或多个所述检测能力所针对的业务系统的配置参数并生成检测规则；
    第二配置子模块，用于：提供代码配置功能，通过所述代码配置功能获取第二类用户提交的描述检测规则的脚本文件，基于所述脚本文件生成检测规则；
    第三配置子模块，用于：提供服务提供接口，通过所述服务提供接口获取第二类用户提交的检测规则并存储。
  5. 根据权利要求1所述的系统，所述目标变更事件在执行之前查找出的当前匹配的检测规则，包括：
    用于检测是否满足目标变更事件的发布条件的检测规则，所述发布条件包括如下任一：目标变更事件的风险等级小于预设的等级阈值的条件；其中，所述等级阈值与当前时刻下目标业务系统的业务量相关，和/或与当前时刻是否在指定日期内相关；所述目标业务系统是所述目标变更事件所变更的业务系统；或，
    用于检测参数配置类型的目标变更事件中所配置的目标参数的参数值是否通过合法性校验的检测规则。
  6. 根据权利要求1所述的系统，每一次执行所述变更事件之后，根据所述目标属性值从已存储的第一检测规则中查找出当前匹配的检测规则，包括：
    对比在目标变更事件发布前后的报错数量和/或报错类型，根据对比结果确定更新后的应用是否发生异常的报错规则；
    对比在目标变更事件发布前后目标业务系统与关联业务系统之间传递的业务数据，根据业务数据的对比结果确定更新后的应用是否发生异常的链路规则；其中，所述目标业务系统是所述应用所属的业务系统，所述关联业务系统是所述目标业务系统的上游和/或下游的业务系统。
  7. 根据权利要求1所述的系统，所述多个预设执行环境包括：预发执行环境、灰度发布环境和正式发布环境；
    所述预设的生命周期管理顺序依次为预发执行环境、灰度发布环境和正式发布环境。
  8. 一种变更风险防控方法，所述方法包括：
    存储多条检测规则，所述检测规则的获取方式包括：提供预定义的用于描述变更事件的多个属性，以及接收第一类用户配置的检测规则及对所述检测规则适用的属性配置的属性值；
    提供所述多个属性，并接收第二类用户的变更发布请求，所述变更发布请求携带所述第二类用户对目标变更事件的属性的目标属性值；
    在多个预设执行环境中，按预设的生命周期管理顺序多次执行所述目标变更事件；其中，在每一次执行所述目标变更事件之前，至少根据所述目标属性值从已存储的检测规则中查找出当前匹配的检测规则并执行，以确定是否可执行所述目标变更事件；以及每一次执行所述目标变更事件之后，至少根据所述目标属性值从已存储的检测规则中查找出当前匹配的检测规则并执行，以确定所述目标变更事件是否导致异常。
  9. 一种电子设备，所述电子设备包括：
    处理器；
    用于存储处理器可执行指令的存储器；
    其中，所述处理器调用所述可执行指令时实现权利要求8所述的方法。
  10. 一种计算机可读存储介质，所述计算机可读存储介质上存储有若干计算机指令，所述计算机指令被执行时执行权利要求8所述的方法。
PCT/CN2023/119954 2022-10-24 2023-09-20 变更风险防控系统、方法、电子设备及存储介质 WO2024087949A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211305683.2A CN115904938B (zh) 2022-10-24 2022-10-24 变更风险防控系统、方法、电子设备及存储介质
CN202211305683.2 2022-10-24

Publications (1)

Publication Number Publication Date
WO2024087949A1 true WO2024087949A1 (zh) 2024-05-02

Family

ID=86475089

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/119954 WO2024087949A1 (zh) 2022-10-24 2023-09-20 变更风险防控系统、方法、电子设备及存储介质

Country Status (2)

Country Link
CN (2) CN115904938B (zh)
WO (1) WO2024087949A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115904938B (zh) * 2022-10-24 2023-08-01 支付宝(杭州)信息技术有限公司 变更风险防控系统、方法、电子设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109241113A (zh) * 2018-08-31 2019-01-18 阿里巴巴集团控股有限公司 风险检查方法及系统
US20190243742A1 (en) * 2018-02-02 2019-08-08 Bank Of America Corporation Smart tool for enterprise-wide software integration and deployment
CN112184235A (zh) * 2020-09-04 2021-01-05 支付宝(杭州)信息技术有限公司 风控数据变更方法和装置
CN112784273A (zh) * 2021-02-10 2021-05-11 中国工商银行股份有限公司 一种sql风险识别方法、装置及设备
CN114238993A (zh) * 2021-12-23 2022-03-25 建信金融科技有限责任公司 风险检测方法、装置、设备及介质
CN115904938A (zh) * 2022-10-24 2023-04-04 支付宝(杭州)信息技术有限公司 变更风险防控系统、方法、电子设备及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190243742A1 (en) * 2018-02-02 2019-08-08 Bank Of America Corporation Smart tool for enterprise-wide software integration and deployment
CN109241113A (zh) * 2018-08-31 2019-01-18 阿里巴巴集团控股有限公司 风险检查方法及系统
CN112184235A (zh) * 2020-09-04 2021-01-05 支付宝(杭州)信息技术有限公司 风控数据变更方法和装置
CN112784273A (zh) * 2021-02-10 2021-05-11 中国工商银行股份有限公司 一种sql风险识别方法、装置及设备
CN114238993A (zh) * 2021-12-23 2022-03-25 建信金融科技有限责任公司 风险检测方法、装置、设备及介质
CN115904938A (zh) * 2022-10-24 2023-04-04 支付宝(杭州)信息技术有限公司 变更风险防控系统、方法、电子设备及存储介质

Also Published As

Publication number Publication date
CN115904938B (zh) 2023-08-01
CN117215948A (zh) 2023-12-12
CN115904938A (zh) 2023-04-04

Similar Documents

Publication Publication Date Title
US10318412B1 (en) Systems, methods, and apparatus for dynamic software generation and testing
US10901727B2 (en) Monitoring code sensitivity to cause software build breaks during software project development
US11829365B2 (en) Systems and methods for data quality monitoring
US10310968B2 (en) Developing software project plans based on developer sensitivity ratings detected from monitoring developer error patterns
De Leoni et al. Data-aware process mining: discovering decisions in processes using alignments
Theisen et al. Approximating attack surfaces with stack traces
Padgham et al. Model-based test oracle generation for automated unit testing of agent systems
US20140013308A1 (en) Application Development Environment with Services Marketplace
US20140013306A1 (en) Computer Load Generator Marketplace
WO2024087949A1 (zh) 变更风险防控系统、方法、电子设备及存储介质
CN113946499A (zh) 一种微服务链路跟踪及性能分析方法、系统、设备及应用
CN113282514B (zh) 问题数据的处理方法、装置、计算机设备和存储介质
US20230199028A1 (en) Techniques for automated capture and reporting of user-verification metric data
CN115952081A (zh) 一种软件测试方法、装置、存储介质及设备
US20140316926A1 (en) Automated Market Maker in Monitoring Services Marketplace
Sebu et al. Business activity monitoring solution to detect deviations in business process execution
CN115496480A (zh) 数据检验方法、系统及相关设备
CN110008098B (zh) 评估业务流程中的节点的运行状况的方法和装置
CN113590484A (zh) 算法模型服务测试方法、系统、设备及存储介质
Kravchenko et al. Complex Dynamic Method of Web Applications Verification by the Criterion of Time Minimization
JP2019194818A (ja) ソフトウェア不具合予測装置
KR102062818B1 (ko) 게임 점검 관리 방법 및 장치
CN114386743A (zh) 一种resar性能工程的性能分析方法及系统
WO2024076253A1 (ru) Способ и система управления модельным риском
CN116361170A (zh) 大数据测试方法、装置、服务器及存储介质