CN113296836B - Method for training model, test method, device, electronic equipment and storage medium - Google Patents

Method for training model, test method, device, electronic equipment and storage medium Download PDF

Info

Publication number
CN113296836B
CN113296836B CN202110639812.0A CN202110639812A CN113296836B CN 113296836 B CN113296836 B CN 113296836B CN 202110639812 A CN202110639812 A CN 202110639812A CN 113296836 B CN113296836 B CN 113296836B
Authority
CN
China
Prior art keywords
item
historical
risk
project
characteristic data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110639812.0A
Other languages
Chinese (zh)
Other versions
CN113296836A (en
Inventor
农华莲
韩照光
李梦阳
黄佳鑫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202110639812.0A priority Critical patent/CN113296836B/en
Publication of CN113296836A publication Critical patent/CN113296836A/en
Application granted granted Critical
Publication of CN113296836B publication Critical patent/CN113296836B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/70—Software maintenance or management
    • G06F8/77—Software metrics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
    • G06F21/577—Assessing vulnerabilities and evaluating computer system security
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2221/00—Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F2221/03—Indexing scheme relating to G06F21/50, monitoring users, programs or devices to maintain the integrity of platforms
    • G06F2221/033—Test or assess software

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Hardware Design (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The disclosure discloses a model training method, a model testing method, a model training device, electronic equipment and a storage medium, and relates to the field of artificial intelligence, in particular to the field of big data. The specific implementation scheme is as follows: processing the historical risk characteristic data set to obtain historical item characteristic data corresponding to each historical item, wherein the historical risk characteristic data set comprises a plurality of historical risk characteristic data, and the historical risk characteristic data are related to item stages; and training the classifier model by using a historical item feature data set to obtain an item risk evaluation model, wherein the historical item feature data set comprises historical item feature data of a plurality of historical items.

Description

Method for training model, testing method, device, electronic equipment and storage medium
Technical Field
The present disclosure relates to the field of artificial intelligence technology, and more particularly, to the field of big data.
Background
To ensure the quality of the project, the project needs to be tested. And as the number of items increases, the pressure of the test also increases.
In order to reduce the testing pressure, a mode of improving the testing efficiency can be adopted, namely, the project risk evaluation can be carried out on the project to obtain a project risk evaluation result, and whether the project is tested or not is determined according to the project risk evaluation result.
Disclosure of Invention
The disclosure provides a method for training a model, a testing method, a testing device, an electronic device and a storage medium.
According to an aspect of the present disclosure, there is provided a method for training a project risk assessment model, comprising: processing a historical risk characteristic data set to obtain historical item characteristic data corresponding to each historical item, wherein the historical risk characteristic data set comprises a plurality of historical risk characteristic data, and the historical risk characteristic data are related to item stages; and training a classifier model by using a historical item feature data set to obtain the item risk evaluation model, wherein the historical item feature data set comprises historical item feature data of a plurality of historical items.
According to another aspect of the present disclosure, there is provided a test method including: acquiring project characteristic data corresponding to a target project; and inputting the item feature data corresponding to the target item into an item risk evaluation model, which is trained by the method described above, to obtain a model prediction result for the target item.
In another aspect of the present disclosure, an apparatus for training a project risk assessment model is provided, comprising: the system comprises a first processing module, a second processing module and a third processing module, wherein the first processing module is used for processing a historical risk characteristic data set to obtain historical item characteristic data corresponding to each historical item, the historical risk characteristic data set comprises a plurality of historical risk characteristic data, and the historical risk characteristic data are related to item stages; and a training module, configured to train a classifier model by using a historical item feature data set to obtain the item risk evaluation model, where the historical item feature data set includes historical item feature data of a plurality of the historical items.
According to another aspect of the present disclosure, there is provided a prediction apparatus including: the second acquisition module is used for acquiring project characteristic data corresponding to the target project; and an obtaining module configured to input the item feature data corresponding to the target item into an item risk evaluation model, which is trained by the apparatus, to obtain a model prediction result for the target item.
According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method.
According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method as described above.
According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method as described above.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 schematically illustrates an exemplary system architecture to which the methods and apparatus for training a project risk assessment model may be applied, according to an embodiment of the present disclosure;
FIG. 2 schematically illustrates a flow diagram of a method for training a project risk assessment model, in accordance with an embodiment of the present disclosure;
FIG. 3 schematically illustrates a flow chart for processing historical risk characteristic data sets to obtain historical item characteristic data corresponding to each historical item, according to an embodiment of the disclosure;
FIG. 4 schematically illustrates a schematic diagram of a training process for training a project risk assessment model, according to an embodiment of the present disclosure;
FIG. 5 schematically shows a flow chart of a prediction method according to an embodiment of the present disclosure;
FIG. 6 schematically illustrates model prediction and model update using a project risk assessment model according to an embodiment of the disclosure;
FIG. 7 schematically illustrates a block diagram of an apparatus for training a project risk assessment model, in accordance with an embodiment of the present disclosure;
FIG. 8 schematically shows a block diagram of a prediction apparatus according to an embodiment of the present disclosure; and
fig. 9 shows a block diagram of an electronic device that may be adapted for use with the method described above according to an embodiment of the disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
With the increase of projects and the larger manpower ratio for developing and testing, the testing pressure is larger and larger. In order to reduce the pressure of the test, a mode of improving the test efficiency can be adopted. The improvement of the testing efficiency can be realized through reasonable test grading cutting, namely, the project risk evaluation can be carried out on the project to obtain the project risk evaluation result, and whether the project is tested or not is determined according to the project risk evaluation result. Namely, aiming at the project with no risk, the risk evaluation result of the project can be free from testing, so that the testing labor is saved and the testing efficiency is improved.
The project risk evaluation can be performed on the project by acquiring the requirement change category and the code quality analysis result and performing the project risk evaluation on the project according to the requirement change category and the code quality analysis result.
In the process of implementing the concept disclosed herein, it is found that the parameters affecting the project risk include other parameters, such as parameters related to the user, in addition to the requirement change type and the code quality, so that the accuracy of the project risk evaluation result obtained by the above method is not high, and the improvement of the test performance is further affected.
In order to solve the above problems, it is found that a project risk evaluation model trained by using feature data capable of describing project risks more comprehensively and accurately may be used to evaluate the project risks. Therefore, the project risk evaluation model for evaluating the project risk is obtained by acquiring the feature data capable of comprehensively and accurately describing the project risk, processing the features, associating the features with the project, and training the classifier model by using the features associated with the project as the training sample.
Based on the foregoing, embodiments of the present disclosure provide a method, prediction method, apparatus, electronic device, non-transitory computer-readable storage medium storing computer instructions, and computer program product for training a project risk evaluation model. The method for training a project risk assessment model comprises the following steps: and processing the historical risk characteristic data set to obtain historical item characteristic data corresponding to each historical item, wherein the historical risk characteristic data set comprises a plurality of historical risk characteristic data, the historical risk characteristic data are related to item stages, and training a classifier model by using the historical item characteristic data set to obtain an item risk evaluation model, wherein the historical item characteristic data set comprises the historical item characteristic data of a plurality of historical items.
FIG. 1 schematically illustrates an exemplary system architecture 100 to which the methods and apparatus for training a project risk assessment model may be applied, according to embodiments of the present disclosure.
It should be noted that fig. 1 is only an example of a system architecture to which the embodiments of the present disclosure may be applied to help those skilled in the art understand the technical content of the present disclosure, and does not mean that the embodiments of the present disclosure may not be applied to other devices, systems, environments or scenarios. For example, in another embodiment, an exemplary system architecture to which the method and apparatus for training a project risk evaluation model may be applied may include a terminal device, but the terminal device may implement the method and apparatus for training a project risk evaluation model provided by the embodiments of the present disclosure without interacting with a server.
As shown in fig. 1, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. Network 104 is the medium used to provide communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired and/or wireless communication links, and so forth.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 may have installed thereon various communication client applications, such as a knowledge reading application, a web browser application, a search application, an instant messaging tool, a mailbox client, and/or social platform software, etc. (by way of example only).
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 105 may be a server providing various services, such as a background management server (for example only) providing support for content browsed by the user using the terminal devices 101, 102, 103. The background management server may analyze and perform other processing on the received data such as the user request, and feed back a processing result (e.g., a webpage, information, or data obtained or generated according to the user request) to the terminal device.
It should be noted that the method for training the risk assessment model of the project provided by the embodiment of the present disclosure may be generally executed by the terminal device 101, 102, or 103. Correspondingly, the device for training the project risk evaluation model provided by the embodiment of the disclosure may also be disposed in the terminal device 101, 102, or 103.
Alternatively, the methods for training a project risk assessment model provided by embodiments of the present disclosure may also be generally performed by the server 105. Accordingly, the apparatus for training the risk assessment model of the project provided by the embodiments of the present disclosure may be generally disposed in the server 105. The method for training a project risk assessment model provided by the embodiments of the present disclosure may also be performed by a server or server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Correspondingly, the apparatus for training the project risk evaluation model provided by the embodiment of the present disclosure may also be disposed in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and/or the server 105.
For example, the server 105 processes a historical risk feature data set including historical item feature data for a plurality of historical items to obtain historical item feature data corresponding to each historical item, the historical risk feature data being associated with an item stage, and trains a classifier model using the historical item feature data set to obtain an item risk evaluation model. Or a server or server cluster capable of communicating with the terminal devices 101, 102, 103 and/or the server 105 trains classifier models with historical project feature data sets to obtain project risk assessment models.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
FIG. 2 schematically shows a flow diagram of a method 200 for training a project risk assessment model according to an embodiment of the disclosure.
As shown in fig. 2, the method includes operations S210 to S220.
In operation S210, a historical risk feature data set is processed to obtain historical item feature data corresponding to each historical item, where the historical risk feature data set includes a plurality of historical risk feature data, and the historical risk feature data is related to an item stage.
In operation S220, the classifier model is trained using a historical item feature data set to obtain an item risk evaluation model, wherein the historical item feature data set includes historical item feature data of a plurality of historical items
According to embodiments of the present disclosure, historical risk profile may refer to profile that is capable of characterizing project risk. The plurality of historical risk profiles comprised by the historical risk profile dataset may be from respective project phases, i.e. the historical risk profile dataset comprises historical risk profiles from respective project phases. The project phase may include at least one of a project requirements phase, a project development phase, a project testing phase, and a project online phase. The historical risk profile data set includes historical risk profile data for each historical item.
According to the embodiment of the present disclosure, the historical item feature data may be understood as data obtained by associating the historical risk feature data with the historical items, that is, for a certain historical item, if the historical risk feature data associated with the historical item is determined from the historical risk feature data set, the historical risk feature data associated with the historical item may be referred to as the historical item feature data. The historical item feature data may be one-dimensional or multi-dimensional. Each dimension of the historical item feature data may refer to a historical risk feature data.
According to embodiments of the present disclosure, the classifier model may include a decision tree model, a logistic regression model, or a neural network model. The type of the classifier model may be configured according to actual business requirements, and is not limited herein.
According to the embodiment of the disclosure, the historical risk characteristic data set can be obtained and processed to obtain the historical item characteristic data corresponding to each historical item. The historical risk characteristic data may have an association relationship with the item identifier, and each historical item may be characterized by the item identifier, and thus, processing the historical risk characteristic data set to obtain the historical item characteristic data corresponding to each historical item may include: and determining historical item feature data corresponding to each historical item from the historical risk feature data set according to the item identification associated with the historical risk feature data.
According to the embodiment of the disclosure, after the historical item feature data corresponding to each historical item is obtained, the classifier model can be trained by using the historical item feature data set, so as to obtain an item risk evaluation model for evaluating the item risk. The historical item feature data set may include a plurality of historical item feature data, each historical item feature data corresponding to a historical item.
According to the embodiment of the disclosure, historical item feature data corresponding to each historical item is obtained by processing the historical risk feature data set, and the classifier model is trained by using the historical item feature data set to obtain the item risk evaluation model. Since the historical risk profile data included in the historical risk profile data set is correlated with the project stage, the historical project profile data of each historical project obtained based on the historical risk profile data set describes the risk of the project more comprehensively and accurately. Therefore, the project risk evaluation model obtained by training based on the historical project characteristic data set can evaluate the project risk more accurately, and further improves the testing efficiency, so that the technical problem that the accuracy of the project risk evaluation result is not high, and the improvement of the testing efficiency is influenced is at least partially solved.
According to an embodiment of the disclosure, training a classifier model by using a historical item feature data set to obtain an item risk evaluation model may include the following operations.
And inputting the historical item feature data corresponding to each historical item into the classifier model to obtain a predicted risk evaluation result corresponding to each historical item. And obtaining an output value by using the predicted risk evaluation result and the real risk evaluation result corresponding to each historical item based on the loss function. And adjusting the model parameters of the classifier model according to the output value until the output value is converged. And determining the classifier model obtained under the condition that the output value is converged as a project risk evaluation model.
According to embodiments of the present disclosure, the risk assessment results may include that the project is at risk or that the project is not at risk. In order to train the classifier model, a real risk assessment result corresponding to each historical item may be obtained, and the real risk assessment result may be understood as a real risk situation of the item.
According to the embodiment of the disclosure, the historical item feature data corresponding to each historical item can be input into the classifier model to obtain a probability value, the probability value is compared with a preset probability threshold, and a predicted risk evaluation result corresponding to each historical item is determined according to the comparison result. The preset probability threshold may be configured according to actual service requirements, and is not limited herein. For example, if the requirement for prediction accuracy is high, the preset probability threshold may be set to a large value. If the recall rate requirement is high, the preset probability threshold value can be set to be a small value. Alternatively, the preset probability threshold may be 0.5.
According to the embodiment of the disclosure, the predicted risk evaluation result and the real risk evaluation result corresponding to each historical item are input into a loss function, and an output value is obtained. After obtaining the output value, the model parameters of the classifier model may be adjusted according to the output value, and the above operation of determining the output value may be repeatedly performed until the output value converges. The model parameters may include one or more.
According to the embodiment of the disclosure, the historical item feature data may be one-dimensional or multi-dimensional feature data, the feature data of each dimension may have a corresponding model parameter, and whether the feature data of each dimension in the historical item feature data represents a positive feature or a negative feature may be determined according to the positive or negative of the model parameter, that is, for the feature data of a dimension, if the model parameter corresponding to the feature data of the dimension is a positive value, it may be stated that the feature data of the dimension represents a positive feature. If the model parameter corresponding to the feature data for that dimension is negative, then it can be stated that the feature data for that dimension characterizes a negative-going feature. In the subsequent process of utilizing the project risk evaluation model to evaluate the project risk of the project, the supplementary test information can be given according to the numerical value of the model parameter. For the project with the project risk, targeted test manpower distribution and test cutting can be performed according to the project risk evaluation result and the feature data corresponding to each dimension, for example, the feature data part with the dimension with higher risk is subjected to the supplementary test according to the supplementary test information, and the feature data part with the dimension with lower risk can be cut, so that the test manpower is effectively utilized.
According to the embodiments of the present disclosure, the classifier model is taken as a logistic regression model as an example for explanation.
The logistic regression model can be characterized by the following formula (1).
Figure BDA0003105708430000081
Wherein the content of the first and second substances,
Figure BDA0003105708430000082
x＝[x0，x1，......，xn-1，xn]，θ＝[θ0，θ1，......，θn-1，θn]. n characterizes the dimensionality of the historical item feature data. x is a radical of a fluorine atomiCharacterizing feature data of the ith dimension. Theta.theta.iCharacterizing the model parameters corresponding to the characteristic data of the ith dimension.
The method shown in fig. 2 is further described with reference to fig. 3-4 in conjunction with specific embodiments.
Fig. 3 schematically illustrates a flow chart of processing a historical risk profile data set to obtain historical item profile data 300 corresponding to each historical item according to an embodiment of the disclosure. The historical risk profile is associated with an item identification.
As shown in fig. 3, the method includes operations S311 to S314.
In operation S311, user risk feature data corresponding to the historical item characterized by the item identifier is determined from the historical risk feature data set according to the item identifier and the user identifier associated with the item identifier.
In operation S312, module risk feature data corresponding to the historical item characterized by the item identifier is determined from the historical risk feature data set according to the item identifier and the branch code identifier associated with the item identifier.
In operation S313, item-level risk characteristic data corresponding to the historical item characterized by the item identification is determined from the historical risk characteristic data set according to the item identification.
In operation S314, historical item feature data corresponding to the historical item characterized by the item identifier is obtained according to the user risk feature data, the module risk feature data, and the item level risk feature data corresponding to the historical item characterized by the item identifier.
According to embodiments of the present disclosure, user risk profile may refer to risk profile associated with a user. Module risk profile may refer to risk profile associated with a module, module risk profile branch risk profile, and/or module thousand row error rate profile. Item level risk profile may refer to profile at the item level. The users may include users associated with various project phases, which may include, for example, product managers, development engineers, and quality assurance engineers.
According to embodiments of the present disclosure, historical risk profile data may be associated with an item identification. The user risk characteristic data may be associated with a user identification, which may be associated with an item identification. A branch code identification may be associated with the module risk profile data and a branch code identification may be associated with the item identification. The item-level risk profile may be associated with an item identification. Thus, historical item feature data corresponding to the historical item characterized by each item identifier can be determined from the historical risk feature data set according to the item identifier and the related identifier associated with the item identifier.
According to an embodiment of the present disclosure, determining, from the historical risk feature data set, historical item feature data corresponding to the historical item characterized by each item identifier according to the item identifier and the related identifier associated with the item identifier may include: for each item identifier, user risk characteristic data corresponding to the historical item characterized by the item identifier may be determined from the historical risk characteristic data set according to the item identifier and the user identifier associated with the item identifier. Module risk characteristic data corresponding to the historical item characterized by the item identification can be determined from the historical risk characteristic data set according to the item identification and the branch code identification corresponding to the item identification. From the item identification, item-level risk profile data corresponding to the historical item characterized by the item identification may be determined from the historical risk profile data set.
The user identifier can be associated with the item identifier, and the user identifier can be associated with the user risk feature data, so that the user risk feature data corresponding to the user identifier can be determined from the historical risk feature data set according to the user identifier, and the user risk feature data corresponding to the user identifier is associated with the historical item represented by the item identifier through the item identifier associated with the user identifier. Since the branch code identification can be associated with the project identification and the branch code identification can be associated with the module risk characteristic data, the module risk characteristic data corresponding to the branch code identification can be determined from the historical risk characteristic data set according to the branch code identification, and the module risk characteristic data corresponding to the branch code identification can be associated with the historical project characterized by the project identification through the project identification associated with the branch code identification. Since the item identification may be associated with the item-level risk characteristic data, the item-level risk characteristic data corresponding to the item identification may be determined from the historical risk characteristic data set according to the item identification.
According to the embodiments of the present disclosure, the determination manner of the historical item feature data is only an exemplary embodiment, but is not limited thereto, and may also include a determination manner known in the art as long as the determination of the historical item feature data can be achieved.
According to an embodiment of the present disclosure, the method for training a risk assessment model of a project may further include the following operations.
An original risk profile dataset is obtained from a target management tool. And processing the original risk characteristic data set to obtain the historical risk characteristic data set.
According to an embodiment of the present disclosure, the raw risk profile data set may comprise a plurality of raw risk profile data. The plurality of raw risk profiles comprised by the raw risk profile dataset may be from various project phases, i.e. the raw risk profile dataset comprises historical risk profiles from various project phases. The raw risk profile data set includes raw risk profile data for each historical item. The relationship between the raw risk profile and the historical risk profile can be understood as: a historical risk profile may be derived based on one or more of the raw risk profiles.
According to embodiments of the present disclosure, a target management tool may refer to a tool that is capable of storing raw risk feature data sets. The target management tool may be matched to the raw risk profile, i.e. the target management tool may comprise a tool capable of acquiring raw risk profile data for various project phases. For example, the target management tool may include at least one of a card management tool, a code management tool, a static scan code tool, a module traffic monitoring tool, a coverage acquisition tool, and an Agile tool. The card management tool may include at least one of a demand card, a development card, and an error rate card.
According to embodiments of the present disclosure, a code management tool may be used to obtain change code for a branch. The static scan code tool may be configured to statically scan code for a target number of errors, which may indicate a number of errors that is greater than or equal to a threshold number of errors. A module flow monitoring tool may be used to monitor the flow of the module. A coverage acquisition tool may be used to acquire coverage associated with the test. The Agile tool may be configured to obtain a commit number and a contrast number in case an operation of detecting a code commit is triggered, so as to process information related to a code change.
According to an embodiment of the present disclosure, at least one of an item identifier, an item requirement, an item type, an item level, a user identifier, and an item online time of an item may be stored to a requirement card. The user identification may include at least one of a product manager identification, a research and development engineer identification, and a quality assurance identification. The branch code identification and the project identification may be stored to a development card. The development engineer identification and branch code identification may be stored to an error rate card. The development card may be associated with the code where the code is submitted in a code management tool.
According to the embodiment of the disclosure, after the original risk characteristic data set is obtained, the original risk characteristic data set may be processed to obtain each historical risk characteristic data included in the historical risk characteristic data set.
FIG. 4 schematically shows a schematic diagram of a training process 400 for training a project risk assessment model, according to an embodiment of the present disclosure.
As shown in fig. 4, the raw risk feature data set 401 includes 100 raw risk feature data, namely, raw risk feature data 4001. The historical risk signature data set includes 80 historical risk signature data, namely, historical risk signature data 4201, a. The historical item feature data set includes historical item feature data for 30 historical items, i.e., historical item feature data 4301, historical item feature data 4330. Each history item has corresponding history item feature data.
According to an embodiment of the present disclosure, the original risk feature dataset 401 is processed to obtain a historical risk feature dataset 402. For each historical item, according to the item identifier and the user identifier associated with the item identifier, determining user risk feature data corresponding to the historical item characterized by the item identifier from the historical risk feature data set 402, according to the item identifier and the branch code identifier associated with the item identifier, determining module risk feature data corresponding to the historical item characterized by the item identifier from the historical risk feature data set 402, according to the item identifier, determining item level risk feature data corresponding to the historical item characterized by the item identifier from the historical risk feature data set 402, and splicing the user risk feature data, the module risk feature data and the item level risk feature data corresponding to the historical item characterized by the item identifier to obtain historical item feature data corresponding to the historical item characterized by the item identifier, that is, historical project characteristic data 4301, historical project characteristic data 4330 are obtained.
According to the embodiment of the disclosure, a classifier model 404 is trained by using a historical item feature data set 403, resulting in an item risk evaluation model 405.
According to an embodiment of the present disclosure, the historical risk profile data is associated with at least one project phase as follows: the method comprises a project requirement stage, a project development stage, a project extraction stage, a project testing stage and a project online stage, wherein historical risk characteristic data corresponding to the project requirement stage comprise at least one of the following items: the system comprises product manager reliability feature data, project grading feature data and project setting feature data. Wherein the historical risk characteristic data corresponding to the project development phase includes at least one of: code development cycle characteristic data, code change line number characteristic data, code change method number characteristic data, code change file number characteristic data, code error rate characteristic data, interface related characteristic data associated with a code change method, code static vulnerability characteristic data and code change method complexity characteristic data. Wherein the historical risk characteristic data corresponding to the project extraction stage includes at least one of: incremental coverage characterization data, extracted throughput characterization data, and extracted number of remaining problems characterization data. Wherein, the historical risk characteristic data corresponding to the project testing stage comprises at least one of the following items: automated regression pass rate feature data, interface related feature data associated with the uncovered change method, and quality assurance reliability feature data. The historical risk characteristic data corresponding to the project online stage comprise online problem quantity characteristic data.
According to an embodiment of the present disclosure, the item setting feature data may refer to feature data that distinguishes from other items. The project setting characteristic data may include at least one of joint tone class characteristic data, compatibility class characteristic data, and configuration class characteristic data. The joint tone type feature data may refer to feature data that needs to be completed cooperatively. Compatibility class feature data may refer to feature data that requires the addition of compatibility tests. Configuration class feature data may refer to feature data that may need to be manually tested or feature data that is less altered.
According to an embodiment of the present disclosure, the interface-related feature data associated with the code change method may include at least one of a number of interfaces affected by the code change method, a number of third party interfaces affected by the code change method, and a number of target interfaces affected by the code change method. A target interface may refer to an interface for which the amount of requests is greater than or equal to a threshold amount of requests. The code error rate characterization data may refer to the thousands of lines of code error rate characterization data of a development engineer.
According to embodiments of the present disclosure, the incremental coverage feature data may include test coverage for manual testing, test coverage for automated testing, or combined test coverage for both manual and automated testing. The test pass rate feature data may refer to the pass rate of the test case.
According to an embodiment of the present disclosure, the interface-related characteristic data associated with the uncovered change method may include at least one of a number of interfaces affected by the uncovered code change method, a number of third party interfaces affected by the uncovered code change method, and a number of target interfaces affected by the uncovered code change method. A target interface may refer to an interface for which the amount of requests is greater than or equal to a threshold amount of requests.
According to the embodiment of the present disclosure, since the historical item feature data may be understood as data obtained by associating the historical risk feature data with the historical item, the historical item feature data may also include at least one item of feature data as described above.
According to embodiments of the present disclosure, the user risk profile may include at least one of a product manager reliability profile, a code error rate profile, and a quality assurance reliability profile. The module risk profile may include branch risk profile and/or module thousand run error rate profile. The branch risk profile may include at least one of code development cycle profile, code change line number profile, code change method number profile, code change file number profile, code error rate profile, interface related profile associated with a code change method, code static vulnerability profile, code change method complexity profile, and interface related profile associated with an uncovered change method. The project level risk profile may include at least one of project rating profile, project setup profile, extraction pass rate profile, extraction left-over problem quantity profile, and automated regression pass rate profile.
According to the embodiment of the disclosure, the number of times of change of the demand can be acquired from the project information included in the demand card, and the reliability of a product research and development manager is determined according to the number of times of change of the demand. The card management tool can be used for determining the number of problems occurring on line and off line of quality assurance, and quality assurance reliability characteristic data can be determined according to the number of problems. The method comprises the steps of recording the starting time under the condition that codes are submitted in a code management tool, recording the ending time after the completion of the submission, and determining the characteristic data of the code development period according to the starting time and the ending time. The method can utilize an Agile tool to obtain a submission number and a comparison number, utilize a code management tool to obtain a change code, and determine code change line number characteristic data and code change file number characteristic data. The method can be used for acquiring a submission number and a comparison number by using an Agile tool, acquiring a change code by using a code management tool, positioning a code change method by using regular matching, and determining the quantity characteristic data of the code change method. In addition, code change conditions can be analyzed by using a command Git Diff of Git (namely, a distributed version control system) to determine code change line number characteristic data, code change file quantity characteristic data and code change method quantity characteristic data.
According to the embodiment of the disclosure, the error information stored in the error rate card can be utilized to determine the error amount of a research and development engineer, obtain the code information submitted by a code management tool, and determine the code error rate characteristic data according to the error amount and the code information.
According to the embodiment of the disclosure, the number of interfaces influenced by the code change method not covered and the number of third party interfaces influenced by the code change method not covered can be determined based on the precise test. The number of target interfaces affected by the uncovered code change method can be obtained by using a module flow monitoring tool.
According to the embodiment of the disclosure, the number of interfaces influenced by the code change method and the number of third party interfaces influenced by the code change method are determined based on the precise test. And acquiring the number of target interfaces influenced by the code change method by using a module flow monitoring tool. And acquiring code static vulnerability characteristic data by using a static code scanning tool, and associating the code static vulnerability characteristic data with the branches.
According to the embodiment of the disclosure, the historical project risk characteristic data can be visually displayed.
According to the embodiment of the disclosure, since the historical project characteristic data of the embodiment of the disclosure is obtained by analyzing the historical risk characteristics of each project stage, such as the project requirement stage, the project development stage, the project extraction and test stage, the project test stage and the project online stage, with the project as a core, the historical project characteristic data can describe the project risk more comprehensively and accurately, so that the project risk can be evaluated more accurately based on the project risk evaluation model obtained by training with the historical project characteristic data set, and the test efficiency is improved, that is, the project without the project risk can be free from testing, and the input of test manpower is saved.
It should be noted that, in the technical solution of the embodiment of the present disclosure, the acquisition, storage, application, and the like of the related user risk characteristic data all meet the regulations of the relevant laws and regulations, and necessary security measures are taken without violating the customs of the public order.
Fig. 5 schematically illustrates a flow chart of a prediction method 500 according to an embodiment of the present disclosure.
As shown in fig. 5, the method includes operations S510 to S520.
In operation S510, item feature data corresponding to a target item is acquired.
In operation S520, project feature data corresponding to the target project is input into a project risk evaluation model, which is trained using the method for training the project risk evaluation model as described above, to obtain a model prediction result for the target project.
According to an embodiment of the present disclosure, a target item may refer to an item for which an item risk assessment is required. The model prediction results may include predicted risk assessment results for the target item.
According to the embodiment of the disclosure, after the item feature data corresponding to the target item is acquired, the target item may be evaluated by using the item risk evaluation model obtained by the method for training the item risk evaluation model according to the embodiment of the disclosure, so as to obtain the model prediction result of the target item.
According to the embodiment of the disclosure, the project characteristic data and/or the model prediction result can be displayed visually.
According to the embodiment of the disclosure, a model prediction result for a target item is obtained by inputting item feature data corresponding to the target item into an item risk evaluation model, wherein the item risk evaluation model is obtained by processing a historical risk feature data set to obtain historical item feature data corresponding to each historical item, and training a classifier model by using the historical item feature data set. Because the historical risk characteristic data included in the historical risk characteristic data set is related to the project stage, the historical project characteristic data of each historical project obtained based on the historical risk characteristic data set can describe the project risk more comprehensively and accurately, and therefore the project risk can be evaluated more accurately based on the project risk evaluation model obtained by training through the historical project characteristic data set, and the testing efficiency is improved.
According to an embodiment of the present disclosure, the model prediction result includes first supplemental test information.
The above prediction method may further include the following operations.
And performing supplementary test on the target item according to the first supplementary test information.
According to an embodiment of the present disclosure, the model prediction result may include a predicted risk evaluation result of the target item and the first supplemental test information. The first supplemental test information may be determined from values of model parameters of the project risk assessment model.
According to the embodiment of the disclosure, the target project can be subjected to the supplementary test according to the first supplementary test information, so that the project risk of the target project can be reduced, and the standard of test approval can be achieved.
According to an embodiment of the present disclosure, the above prediction method may further include the following operations.
And acquiring a manual feedback result aiming at the target project. And executing the target task according to the manual feedback result of the target project.
According to the embodiment of the disclosure, the manual feedback result can be used for confirming whether the model prediction result is correct and/or optimizing the project risk evaluation model. According to the result of the manual feedback of the target project, the target task execution may include: and performing supplementary test on the target project according to the manual feedback result of the target project. Alternatively, the project risk evaluation model is updated according to the manual feedback result and the model prediction result of the target project. Alternatively, the feature dimension that needs to be added is determined according to the result of manual feedback of the target item.
According to the embodiment of the disclosure, the manual feedback result can be visually displayed.
According to an embodiment of the present disclosure, the manual feedback result includes a manual risk evaluation result and second supplemental test information.
And executing the target task according to the manual feedback result of the target project, wherein the following operations can be included.
And in the case that the result of the artificial risk evaluation of the target project is determined to be that the target project is at risk, performing supplementary test on the target project according to the second supplementary test information.
According to the embodiment of the disclosure, if the artificial risk evaluation result of the target project indicates that the target project has risk, the target project can be supplemented according to the second supplementary test information, so that the project risk of the target project can be reduced, and the standard of test approval can be achieved.
According to an embodiment of the present disclosure, the artificial feedback result includes an artificial risk evaluation result and an objective model evaluation parameter.
And executing the target task according to the manual feedback result of the target project, wherein the following operations can be included.
And under the condition that the artificial risk evaluation result of the target project is determined to be inconsistent with the predicted risk evaluation result of the target project, updating the project risk evaluation model so that the updated project risk evaluation model meets the target model evaluation parameters.
According to embodiments of the present disclosure, the target model evaluation parameters may include accuracy and/or recall.
According to the embodiment of the disclosure, if it is determined that the artificial risk evaluation result of the target item is inconsistent with the predicted risk evaluation result of the target item, that is, the artificial risk evaluation result of the target item represents that the target item has a risk but the predicted risk evaluation result represents that the target item does not have a risk, or the artificial risk evaluation result of the target item represents that the target item does not have a risk but the predicted risk evaluation result represents that the target item has a risk, it may be said that the item risk evaluation model needs to be optimized, and in this case, the item risk evaluation model may be updated so that the updated item risk evaluation model satisfies the target model evaluation parameters.
According to an embodiment of the present disclosure, the manual feedback result includes a feature dimension to be supplemented.
And executing the target task according to the manual feedback result of the target project, wherein the following operations can be included.
And acquiring project characteristic data corresponding to the characteristic dimension to be supplemented. And adding the project characteristic data corresponding to the characteristic dimension to be supplemented to the historical risk characteristic data set to obtain a new historical risk characteristic data set. And updating the project risk evaluation model by using the new historical risk characteristic data set.
According to an embodiment of the present disclosure, a feature dimension to be supplemented may refer to project feature data that needs to be supplemented. After the characteristic dimension to be supplemented is determined, project characteristic data corresponding to the characteristic dimension to be supplemented can be obtained, the characteristic data corresponding to the characteristic dimension to be supplemented are added to the historical risk characteristic data set to obtain a new historical risk characteristic data set, the new historical risk characteristic data set is processed to obtain new historical project characteristic data corresponding to each historical project, and the project risk evaluation model is updated by the new historical project characteristic data set.
According to the embodiment of the disclosure, the target item is related to the upgrading of the component class, and the code change is less, so that the predicted risk evaluation result obtained by using the item risk evaluation model may not be at risk for the target item, and therefore, the target item can be free from testing. However, because compatibility tests may need to be performed, the result of the manual risk assessment is that the target item is at risk. Thus, the feature dimension to be supplemented can be increased, i.e., compatibility component class upgrades are added.
According to the embodiment of the disclosure, the project risk evaluation model can continuously optimize the model evaluation parameters through the manual feedback result, for example, the model evaluation parameters can include judgment accuracy and recall rate, and new feature dimensions can be timely identified and supplemented from the manual feedback result.
The method of fig. 5 is further described with reference to fig. 6 in conjunction with specific embodiments.
FIG. 6 schematically shows a schematic diagram of model prediction and model update 600 using a project risk assessment model according to an embodiment of the disclosure.
As shown in fig. 6, item feature data 601 corresponding to a target item is acquired, and the item feature data 601 is input to an item risk evaluation model 602 to obtain a model prediction result 603 for the target item. And further acquiring a manual feedback result 604 for the target project, and updating the project risk evaluation model 602 according to the manual feedback result 604.
FIG. 7 schematically illustrates a block diagram of an apparatus 700 for training a project risk assessment model, according to an embodiment of the present disclosure.
As shown in FIG. 7, an apparatus 700 for training a project risk assessment model may include a first processing module 710 and a training module 720.
The first processing module 710 is configured to process a historical risk feature data set to obtain historical item feature data corresponding to each historical item, where the historical risk feature data set includes a plurality of historical risk feature data, and the historical risk feature data is related to item phases.
And a training module 720, configured to train the classifier model by using a historical item feature data set to obtain an item risk evaluation model, where the historical item feature data set includes historical item feature data of a plurality of historical items.
According to an embodiment of the present disclosure, historical risk profile data is associated with an item identification;
the training module 720 may include a first determining unit, a second determining unit, a third determining unit, and a first obtaining unit.
And the first determining unit is used for determining user risk characteristic data corresponding to the historical item represented by the item identification from the historical risk characteristic data set according to the item identification and the user identification associated with the item identification.
And the second determining unit is used for determining module risk characteristic data corresponding to the historical item represented by the item identification from the historical risk characteristic data set according to the item identification and the branch code identification associated with the item identification.
And the third determining unit is used for determining item level risk characteristic data corresponding to the historical item represented by the item identifier from the historical risk characteristic data set according to the item identifier.
And the first obtaining unit is used for obtaining historical item feature data corresponding to the historical item represented by the item identifier according to the user risk feature data, the module risk feature data and the item level risk feature data corresponding to the historical item represented by the item identifier.
According to an embodiment of the present disclosure, the apparatus 700 for training a risk assessment model of a project may further include a first obtaining module and a second processing module.
The first acquisition module is used for acquiring an original risk characteristic data set from a target management tool.
And the second processing module is used for processing the original risk characteristic data set to obtain a historical risk characteristic data set.
According to an embodiment of the present disclosure, the historical risk profile data is associated with at least one project phase: the method comprises a project requirement stage, a project development stage, a project extraction stage, a project test stage and a project online stage, wherein historical risk characteristic data corresponding to the project requirement stage comprises at least one of the following items: the system comprises product manager reliability feature data, project grading feature data and project setting feature data. Wherein the historical risk profile data corresponding to the project development phase includes at least one of: code development cycle characteristic data, code change line number characteristic data, code change method number characteristic data, code change file number characteristic data, code error rate characteristic data, interface related characteristic data associated with a code change method, code static vulnerability characteristic data and code change method complexity characteristic data. Wherein the historical risk characteristic data corresponding to the project extraction stage includes at least one of: incremental coverage characterization data, extracted throughput characterization data, and extracted number of remaining problems characterization data. Wherein the historical risk characteristic data corresponding to the project testing phase includes at least one of: automated regression pass rate feature data, interface related feature data associated with the uncovered change method, and quality assurance reliability feature data. The historical risk characteristic data corresponding to the project online stage comprise online problem quantity characteristic data.
According to an embodiment of the present disclosure, training module 720 may include
And the second obtaining unit is used for inputting the historical item feature data corresponding to each historical item into the classifier model to obtain a predicted risk evaluation result corresponding to each historical item.
And a third obtaining unit, configured to obtain an output value using the predicted risk evaluation result and the actual risk evaluation result corresponding to each history item based on the loss function.
And the adjusting unit is used for adjusting the model parameters of the classifier model according to the output value until the output value is converged.
A fourth determination unit configured to determine the classifier model obtained when the output value converges as the item risk evaluation model.
Fig. 8 schematically shows a block diagram of a prediction apparatus 800 according to an embodiment of the present disclosure.
As shown in fig. 8, the prediction apparatus 800 may include a second obtaining module 810 and an obtaining module 820.
A second obtaining module 810, configured to obtain item feature data corresponding to the target item.
An obtaining module 820, configured to input the item feature data corresponding to the target item into an item risk evaluation model, so as to obtain a model prediction result for the target item, where the item risk evaluation model is trained by the apparatus 700 for training the item risk evaluation model.
According to an embodiment of the present disclosure, the model prediction result includes first supplemental test information.
The prediction apparatus 800 may further include a test module.
And the testing module is used for performing supplementary testing on the target item according to the first supplementary testing information.
According to an embodiment of the present disclosure, the prediction apparatus 800 may further include a third obtaining module and an executing module.
And the third acquisition module is used for acquiring a manual feedback result aiming at the target project.
And the execution module is used for executing the target task according to the manual feedback result of the target project.
According to an embodiment of the present disclosure, the manual feedback result includes a manual risk evaluation result and second supplemental test information.
The execution module may comprise a supplementary unit.
And the supplementary unit is used for performing supplementary test on the target item according to the second supplementary test information under the condition that the artificial risk evaluation result of the target item is determined to be that the target item is at risk.
According to an embodiment of the present disclosure, the artificial feedback result includes an artificial risk evaluation result and a target model evaluation parameter.
The execution module may include a first update unit.
And the first updating unit is used for updating the project risk evaluation model under the condition that the artificial risk evaluation result of the target project is determined to be inconsistent with the predicted risk evaluation result of the target project, so that the updated project risk evaluation model meets the target model evaluation parameters.
According to an embodiment of the present disclosure, the manual feedback result includes a feature dimension to be supplemented.
The execution module may include an acquisition unit, an addition unit, and a second update unit.
And the acquisition unit is used for acquiring the project characteristic data corresponding to the characteristic dimension to be supplemented.
And the adding unit is used for adding the project characteristic data corresponding to the characteristic dimension to be supplemented to the historical risk characteristic data set to obtain a new historical risk characteristic data set.
And the second updating unit is used for updating the project risk evaluation model by using the new historical risk characteristic data set.
The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described above.
According to an embodiment of the present disclosure, a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method as described above.
According to an embodiment of the disclosure, a computer program product comprising a computer program which, when executed by a processor, implements the method as described above.
Fig. 9 shows a block diagram of an electronic device 900 that may be adapted for use with the method described above according to an embodiment of the disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 9, the apparatus 900 includes a computing unit 901 which can perform various appropriate actions and processes in accordance with a computer program stored in a Read Only Memory (ROM)902 or a computer program loaded from a storage unit 908 into a Random Access Memory (RAM) 903. In the RAM903, various programs and data required for the operation of the device 900 can also be stored. The calculation unit 901, ROM902, and RAM903 are connected to each other via a bus 904. An input/output (I/O) interface 905 is also connected to bus 904.
A number of components in the device 900 are connected to the I/O interface 905, including: an input unit 906 such as a keyboard, a mouse, and the like; an output unit 907 such as various types of displays, speakers, and the like; a storage unit 908 such as a magnetic disk, optical disk, or the like; and a communication unit 909 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 909 allows the device 900 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.
The computing unit 901 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The computing unit 901 performs the various methods and processes described above, such as methods for training a project risk assessment model or predictive methods. For example, in some embodiments, the method for training a project risk assessment model or a predictive method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of a computer program may be loaded onto and/or installed onto device 900 via ROM902 and/or communications unit 909. When loaded into RAM903 and executed by computing unit 901, may perform one or more of the steps of the above-described methods for training a project risk assessment model or predictive methods. Alternatively, in other embodiments, the computing unit 901 may be configured by any other suitable means (e.g., by means of firmware) to execute a method for training a project risk assessment model or a predictive method.
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server combining a blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims (17)

1. A method for training a project risk assessment model, comprising:
processing a historical risk characteristic data set to obtain historical item characteristic data corresponding to each historical item, wherein the historical risk characteristic data set comprises a plurality of historical risk characteristic data, and the historical risk characteristic data are related to item stages; and
training a classifier model by utilizing a historical project characteristic data set to obtain the project risk evaluation model, wherein the historical project characteristic data set comprises historical project characteristic data of a plurality of historical projects;
wherein the historical risk profile is associated with an item identification;
the processing the historical risk characteristic data set to obtain the historical item characteristic data corresponding to each historical item includes:
determining user risk characteristic data corresponding to the historical item represented by the item identifier from the historical risk characteristic data set according to the item identifier and the user identifier associated with the item identifier;
determining module risk characteristic data corresponding to the historical item characterized by the item identifier from the historical risk characteristic data set according to the item identifier and the branch code identifier associated with the item identifier;
determining project level risk characteristic data corresponding to the historical projects characterized by the project identifiers from the historical risk characteristic data set according to the project identifiers; and
and obtaining historical item feature data corresponding to the historical item represented by the item identifier according to the user risk feature data, the module risk feature data and the item level risk feature data corresponding to the historical item represented by the item identifier.
2. The method of claim 1, further comprising:
acquiring an original risk characteristic data set from a target management tool; and
and processing the original risk characteristic data set to obtain the historical risk characteristic data set.
3. The method of claim 1 or 2, wherein the historical risk profile data is associated with at least one of the project phases: a project requirement stage, a project development stage, a project promotion and test stage, a project test stage and a project online stage,
wherein the historical risk characteristic data corresponding to the project demand phase includes at least one of: reliability feature data of a product manager, item classification feature data and item setting feature data;
wherein the historical risk profile data corresponding to the project development phase includes at least one of: code development cycle characteristic data, code change line number characteristic data, code change method number characteristic data, code change file number characteristic data, code error rate characteristic data, interface related characteristic data associated with a code change method, code static vulnerability characteristic data and code change method complexity characteristic data;
wherein the historical risk characteristic data corresponding to the project recommendation phase includes at least one of: incremental coverage rate characteristic data, extraction and measurement of passing rate characteristic data and extraction and measurement of remaining problem quantity characteristic data;
wherein the historical risk characteristic data corresponding to the project testing phase includes at least one of: automated regression pass rate feature data, interface related feature data associated with an uncovered change method, and quality assurance reliability feature data;
and historical risk characteristic data corresponding to the project online stage comprise online problem quantity characteristic data.
4. The method of claim 1 or 2, wherein said training a classifier model using a historical project feature data set to derive a project risk assessment model comprises:
inputting historical item feature data corresponding to each historical item into the classifier model to obtain a predicted risk evaluation result corresponding to each historical item;
obtaining an output value by using a predicted risk evaluation result and a real risk evaluation result corresponding to each historical item based on a loss function;
according to the output value, adjusting the model parameters of the classifier model until the output value is converged; and
and determining the classifier model obtained under the condition that the output value is converged as the project risk evaluation model.
5. A prediction method, comprising:
acquiring project characteristic data corresponding to a target project; and
inputting the project characteristic data corresponding to the target project into a project risk evaluation model to obtain a model prediction result aiming at the target project,
wherein the project risk assessment model is trained using the method according to any one of claims 1 to 4.
6. The method of claim 5, wherein the model prediction result comprises first supplemental test information;
the method further comprises the following steps:
and performing supplementary test on the target item according to the first supplementary test information.
7. The method of claim 5, further comprising:
acquiring a manual feedback result aiming at the target project; and
and executing the target task according to the manual feedback result of the target project.
8. The method of claim 7, wherein the manual feedback results include manual risk assessment results and second supplemental test information;
the executing the target task according to the artificial feedback result of the target project comprises the following steps:
and performing supplementary test on the target item according to the second supplementary test information if the result of the artificial risk evaluation of the target item is determined to be that the target item is at risk.
9. The method of claim 7, wherein the manual feedback results include manual risk assessment results and objective model assessment parameters;
the executing the target task according to the artificial feedback result of the target project comprises the following steps:
and under the condition that the artificial risk evaluation result of the target item is determined to be inconsistent with the predicted risk evaluation result of the target item, updating the item risk evaluation model so that the updated item risk evaluation model meets the target model evaluation parameters.
10. The method of claim 7, wherein the artificial feedback result comprises a feature dimension to be supplemented;
the executing the target task according to the manual feedback result of the target project comprises the following steps:
acquiring project characteristic data corresponding to the characteristic dimension to be supplemented;
adding project characteristic data corresponding to the characteristic dimension to be supplemented to the historical risk characteristic data set to obtain a new historical risk characteristic data set; and
and updating the project risk evaluation model by using the new historical risk characteristic data set.
11. An apparatus for training a project risk assessment model, comprising:
the system comprises a first processing module, a second processing module and a third processing module, wherein the first processing module is used for processing a historical risk characteristic data set to obtain historical item characteristic data corresponding to each historical item, the historical risk characteristic data set comprises a plurality of historical risk characteristic data, and the historical risk characteristic data are related to item stages; and
the training module is used for training a classifier model by utilizing a historical item feature data set to obtain the item risk evaluation model, wherein the historical item feature data set comprises a plurality of historical item feature data of the historical items;
the historical risk characteristic data is associated with an item identifier;
the training module comprises:
a first determining unit, configured to determine, according to the item identifier and a user identifier associated with the item identifier, user risk feature data corresponding to a historical item characterized by the item identifier from the historical risk feature data set;
a second determining unit, configured to determine, from the historical risk characteristic data set, module risk characteristic data corresponding to a historical item characterized by the item identifier according to the item identifier and a branch code identifier associated with the item identifier;
a third determining unit, configured to determine, according to the item identifier, item-level risk feature data corresponding to a historical item characterized by the item identifier from the historical risk feature data set; and
and the obtaining unit is used for obtaining the historical item characteristic data corresponding to the historical item represented by the item identifier according to the user risk characteristic data, the module risk characteristic data and the item level risk characteristic data corresponding to the historical item represented by the item identifier.
12. The apparatus of claim 11, further comprising:
the first acquisition module is used for acquiring an original risk characteristic data set from a target management tool; and
and the second processing module is used for processing the original risk characteristic data set to obtain the historical risk characteristic data set.
13. A prediction apparatus, comprising:
the second acquisition module is used for acquiring project characteristic data corresponding to the target project; and
an obtaining module, configured to input the item feature data corresponding to the target item into an item risk evaluation model to obtain a model prediction result for the target item,
wherein the project risk assessment model is trained using the apparatus of any one of claims 11-12.
14. The apparatus of claim 13, wherein the model prediction result comprises first supplemental test information;
the device further comprises:
and the testing module is used for performing supplementary testing on the target item according to the first supplementary testing information.
15. The apparatus of claim 13, further comprising:
the third acquisition module is used for acquiring a manual feedback result aiming at the target project; and
and the execution module is used for executing the target task according to the artificial feedback result of the target project.
16. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 4 or any one of claims 5 to 10.
17. A non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method of any of claims 1-4 or any of claims 5-10.
CN202110639812.0A 2021-06-08 2021-06-08 Method for training model, test method, device, electronic equipment and storage medium Active CN113296836B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110639812.0A CN113296836B (en) 2021-06-08 2021-06-08 Method for training model, test method, device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110639812.0A CN113296836B (en) 2021-06-08 2021-06-08 Method for training model, test method, device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN113296836A CN113296836A (en) 2021-08-24
CN113296836B true CN113296836B (en) 2022-07-22

Family

ID=77327625

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110639812.0A Active CN113296836B (en) 2021-06-08 2021-06-08 Method for training model, test method, device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113296836B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115277404B (en) * 2022-05-13 2023-06-02 清华大学 Cloud network large-scale change release arrangement method, device, equipment and storage medium

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101556553A (en) * 2009-03-27 2009-10-14 中国科学院软件研究所 Defect prediction method and system based on requirement change
CN110399302A (en) * 2019-07-26 2019-11-01 中国工商银行股份有限公司 Risk Identification Method, device, electronic equipment and the medium of Software Testing Project
CN110413513A (en) * 2019-07-11 2019-11-05 北京百度网讯科技有限公司 Publication quality control system, method and the device of software
CN110826071A (en) * 2019-09-24 2020-02-21 平安科技（深圳）有限公司 Software vulnerability risk prediction method, device, equipment and storage medium
CN111274126A (en) * 2020-01-14 2020-06-12 华为技术有限公司 Test case screening method, device and medium
CN111459828A (en) * 2020-04-07 2020-07-28 中国建设银行股份有限公司 Non-functional test evaluation method and device for software version
CN112379913A (en) * 2020-11-20 2021-02-19 上海复深蓝软件股份有限公司 Software optimization method, device, equipment and storage medium based on risk identification
CN112465011A (en) * 2020-11-25 2021-03-09 平安医疗健康管理股份有限公司 Project risk prediction method and system based on project research and development process

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103208039B (en) * 2012-01-13 2017-05-03 株式会社日立制作所 Method and device for evaluating software project risks
CN108305158B (en) * 2017-12-27 2020-06-09 阿里巴巴集团控股有限公司 Method, device and equipment for training wind control model and wind control
CN110298547A (en) * 2019-05-24 2019-10-01 深圳壹账通智能科技有限公司 Methods of risk assessment, device, computer installation and storage medium

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101556553A (en) * 2009-03-27 2009-10-14 中国科学院软件研究所 Defect prediction method and system based on requirement change
CN110413513A (en) * 2019-07-11 2019-11-05 北京百度网讯科技有限公司 Publication quality control system, method and the device of software
CN110399302A (en) * 2019-07-26 2019-11-01 中国工商银行股份有限公司 Risk Identification Method, device, electronic equipment and the medium of Software Testing Project
CN110826071A (en) * 2019-09-24 2020-02-21 平安科技（深圳）有限公司 Software vulnerability risk prediction method, device, equipment and storage medium
CN111274126A (en) * 2020-01-14 2020-06-12 华为技术有限公司 Test case screening method, device and medium
CN111459828A (en) * 2020-04-07 2020-07-28 中国建设银行股份有限公司 Non-functional test evaluation method and device for software version
CN112379913A (en) * 2020-11-20 2021-02-19 上海复深蓝软件股份有限公司 Software optimization method, device, equipment and storage medium based on risk identification
CN112465011A (en) * 2020-11-25 2021-03-09 平安医疗健康管理股份有限公司 Project risk prediction method and system based on project research and development process

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
BadNets: Evaluating Backdooring Attacks on Deep Neural Networks;Tianyu Gu等;《IEEE Access》;20190411;47230 - 47244 *
基于多核集成学习的跨项目软件缺陷预测;黄琳等;《计算机技术与发展》;20190316;27-31 *
基于机器学习的软件缺陷预测方法研究;张志武;《中国优秀博硕士学位论文全文数据库(博士)信息科技辑》;20200115;I138-13 *
软件项目风险评价研究;付峰等;《河南科学》;20090215(第02期);205-209 *

Also Published As

Publication number Publication date
CN113296836A (en) 2021-08-24

Similar Documents

Publication Publication Date Title
CN108170909B (en) Intelligent modeling model output method, equipment and storage medium
CN107517251B (en) Information pushing method and device
CN109872230B (en) Test method and device of financial data analysis system, medium and electronic equipment
CN113760769A (en) Test case processing method and device, electronic equipment and storage medium
CN109784352A (en) A kind of method and apparatus for assessing disaggregated model
CN113051183A (en) Test data recommendation method and system, electronic device and storage medium
CN113296836B (en) Method for training model, test method, device, electronic equipment and storage medium
US20230196245A1 (en) Method and apparatus for predicting risk, electronic device, computer readable storage medium
CN112966971B (en) Project workload assessment method and device
CN114693116A (en) Method and device for detecting code review validity and electronic equipment
CN115062687A (en) Enterprise credit monitoring method, device, equipment and storage medium
CN114881521A (en) Service evaluation method, device, electronic equipment and storage medium
CN111429257B (en) Transaction monitoring method and device
CN113850072A (en) Text emotion analysis method, emotion analysis model training method, device, equipment and medium
CN113450208A (en) Loan risk change early warning and model training method and device
CN113051479A (en) File processing and recommendation information generation method, device, equipment and storage medium
CN112905469A (en) Method and apparatus for detecting applet capability features
CN116823407B (en) Product information pushing method, device, electronic equipment and computer readable medium
CN117271373B (en) Automatic construction method and device for test cases, electronic equipment and storage medium
CN113536672B (en) Target object processing method and device
CN117634470A (en) Verification method, device, equipment and storage medium
CN115238817A (en) Model training and demand judging method, device, equipment, medium and product
CN117421311A (en) Data verification method, device, equipment and storage medium based on artificial intelligence
CN114417219A (en) Information configuration method and device and electronic equipment
CN114647578A (en) System test method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant