CN115829722A - Training method of credit risk scoring model and credit risk scoring method - Google Patents

Training method of credit risk scoring model and credit risk scoring method Download PDF

Info

Publication number
CN115829722A
CN115829722A CN202211521953.3A CN202211521953A CN115829722A CN 115829722 A CN115829722 A CN 115829722A CN 202211521953 A CN202211521953 A CN 202211521953A CN 115829722 A CN115829722 A CN 115829722A
Authority
CN
China
Prior art keywords
data
training
model
neural network
financial
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211521953.3A
Other languages
Chinese (zh)
Inventor
张欢
李卓阳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Agricultural Bank of China
Original Assignee
Agricultural Bank of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Agricultural Bank of China filed Critical Agricultural Bank of China
Priority to CN202211521953.3A priority Critical patent/CN115829722A/en
Publication of CN115829722A publication Critical patent/CN115829722A/en
Pending legal-status Critical Current

Links

Images

Abstract

The application provides a training method of a credit risk scoring model and a credit risk scoring method. The method comprises the following steps: acquiring data of a plurality of financial indexes and data of a plurality of non-financial indexes of a plurality of enterprises, and generating a multi-dimensional time sequence data matrix according to the data of the plurality of financial indexes and the data of the plurality of non-financial indexes; acquiring an initial scoring model, wherein the initial scoring model comprises a differential autoregressive moving average (ARIMA) model and a Back Propagation (BP) neural network; and taking data in the multi-dimensional time sequence data as training data, respectively training the ARIMA model and the BP neural network, and obtaining a target risk scoring model when a training termination condition is reached, wherein the target risk scoring model is used for generating a risk value of the data to be evaluated based on the input data to be evaluated. The method can solve the problem of how to objectively evaluate the credit risk of the enterprise based on the real enterprise operation data and improve the accuracy of the credit risk evaluation.

Description

Training method of credit risk scoring model and credit risk scoring method
Technical Field
The application relates to the internet financial technology, in particular to a training method of a credit risk scoring model and a credit risk scoring method.
Background
The credit risk is also called financial default risk, and refers to the probability that a lender, debtor or transaction participant cannot fulfill contract terms for various reasons and constitutes a default. When a bank docks an enterprise, the credit risk of the enterprise is generally evaluated in order to avoid default.
By 2020, the proportion of small and medium-sized enterprises registered in China to the whole number of registered industrial and commercial enterprises in China reaches 97%, and 65% of total domestic production value is provided. However, when a bank evaluates the credit risk of a small and medium enterprise, the bank often relies on the review experience of professional reviewers for evaluation, and the professional reviewers cannot make objective credit risk evaluation on the central enterprise due to the stereotyped impression of the small and medium enterprises (e.g., the enterprise has many operating conditions and short operating time).
How to objectively evaluate the credit risk of an enterprise based on real enterprise operation data and improve the accuracy of credit risk evaluation still needs to be solved.
Disclosure of Invention
The application provides a training method of a credit risk scoring model and a credit risk scoring method, which are used for solving the problems of how to objectively evaluate credit risks of enterprises based on real enterprise operation data and how to improve the accuracy of credit risk evaluation.
On the one hand, the application provides a method for objectively evaluating the credit risk of an enterprise based on real enterprise operation data, and the accuracy of the credit risk evaluation is improved, and the method comprises the following steps:
acquiring data of a plurality of financial indexes and data of a plurality of non-financial indexes of a plurality of enterprises, wherein the data of each financial index and the data of each non-financial index carry time information, and generating a multi-dimensional time sequence data matrix according to the data of the plurality of financial indexes and the data of the plurality of non-financial indexes;
acquiring an initial scoring model, wherein the initial scoring model comprises a differential autoregressive moving average (ARIMA) model and a Back Propagation (BP) neural network, the ARIMA model is used for outputting prediction data of each data input to the ARIMA model in time, the BP neural network is used for outputting the weight of each data input to the BP neural network, and the data input to the BP neural network comprises the prediction data;
and taking data in the multi-dimensional time sequence data as training data, respectively training an ARIMA model and a BP neural network, and obtaining a target risk scoring model when a training termination condition is reached, wherein the target risk scoring model is used for generating a risk value of the data to be evaluated based on the input data to be evaluated.
In one embodiment, the training the ARIMA model and the BP neural network with data in the multidimensional time series data as training data to obtain the target risk score model includes:
and taking front F column data in the multi-dimensional time sequence data as input data of an ARIMA model, taking back P column data in the multi-dimensional time sequence data as reference data of output data of the ARIMA model, training the ARIMA model until a training termination condition of the ARIMA model is reached to obtain a target ARIMA model, wherein the training termination condition of the ARIMA model comprises one or more of the following conditions: the difference value between the output data of the ARIMA model and the reference data of the output data of the ARIMA model is within a first preset range, the training time length reaches the preset time length, and the training times reaches the preset times;
acquiring first input data of a BP (back propagation) neural network, which are data with the same time in the multi-dimensional time sequence data, acquiring reference data of which the known weight of the first input data is output data of the BP neural network, and training the BP neural network by using the output data of the ARIMA model as second input data of the BP neural network until a training termination condition of the BP neural network is reached to obtain a target BP neural network, wherein the training termination condition of the BP neural network comprises one or more of the following conditions: the difference value between the output data of the BP neural network and the reference data of the output data of the BP neural network is in a second preset range, the training time length reaches the preset time length, and the training times reaches the preset times;
and obtaining a target risk scoring model based on the target ARIMA model and the target BP neural network, wherein the target risk scoring model is used for generating a risk value of the data to be evaluated according to the data to be evaluated, the predicted data of the data to be evaluated, the weight of the data to be evaluated and the weight of the predicted data of the data to be evaluated.
In one embodiment, the generating a multi-dimensional time series data matrix from the data of the plurality of financial indicators and the data of the plurality of non-financial indicators includes:
clustering data of a plurality of non-financial indexes to obtain time sequence data of one dimension;
respectively taking the data of each financial index as time series data of one dimension to obtain time series data of multiple dimensions;
and generating a multi-dimensional time series data matrix based on the time series data of one dimension obtained from the data of the plurality of non-financial indexes and the time series data of a plurality of dimensions obtained from the data of the plurality of financial indexes.
In one embodiment, the generating the multi-dimensional time series data matrix includes:
generating an initial multi-dimensional time series data matrix based on time series data of one dimension obtained from data of a plurality of non-financial indexes and time series data of a plurality of dimensions obtained from data of a plurality of financial indexes;
processing the initial multi-dimensional time series data matrix based on a sliding window algorithm with a preset acquisition step length to obtain a plurality of data sets;
generating the multi-dimensional time series data matrix based on the plurality of data sets.
In one embodiment, the clustering data of a plurality of non-financial indexes to obtain time series data of a dimension includes:
acquiring a preset classification standard, wherein the preset classification standard comprises M classes, and M is a natural number greater than 1;
respectively taking the randomly selected M pieces of data of the non-financial indexes as M pieces of first central point data;
determining second central point data of a first type according to the data of the plurality of non-financial indexes when the types of the data of the plurality of non-financial indexes are determined to be the first type according to the distance between the data of each non-financial index and the first central point data of the M types;
and repeating the step, wherein the randomly selected M pieces of data of the non-financial indexes are respectively M pieces of first central point data, and the second central point data is determined to be time sequence data of one dimension when the second central point data of the first type is determined to be a fixed value every time.
In another aspect, the present application provides a credit risk scoring method, including:
acquiring data to be evaluated, wherein the data to be evaluated comprises data of a plurality of financial indexes and/or data of a plurality of non-financial indexes;
and inputting the data to be evaluated into the target risk scoring model obtained by training the training method of the credit risk scoring model according to the first aspect, so as to obtain the risk value of the data to be evaluated.
In another aspect, the present application provides a training apparatus for a credit risk scoring model, including:
the system comprises an acquisition module, a time information acquisition module and a time information acquisition module, wherein the acquisition module is used for acquiring data of a plurality of financial indexes and data of a plurality of non-financial indexes of a plurality of enterprises, and generating a multi-dimensional time sequence data matrix according to the data of the financial indexes and the data of the non-financial indexes;
the obtaining module is further configured to obtain an initial scoring model, where the initial scoring model includes a differential autoregressive moving average (ARIMA) model and a Back Propagation (BP) neural network, the ARIMA model is configured to output predicted data of each data input to the ARIMA model over time, the BP neural network is configured to output weights of each data input to the BP neural network, and the data input to the BP neural network includes the predicted data;
and the training module is used for respectively training an ARIMA model and a BP neural network by taking data in the multi-dimensional time sequence data as training data to obtain a target risk scoring model when a training termination condition is reached, and the target risk scoring model is used for generating a risk value of the data to be evaluated based on the input data to be evaluated.
In another aspect, the present application provides a credit risk scoring apparatus, including:
the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring data to be evaluated, and the data to be evaluated comprises data of a plurality of financial indexes and/or data of a plurality of non-financial indexes;
the evaluation module is configured to input the data to be evaluated into the target risk scoring model obtained by training the training method of the credit risk scoring model according to the first aspect, so as to obtain a risk value of the data to be evaluated.
In another aspect, the present application provides an electronic device comprising: a processor, and a memory communicatively coupled to the processor;
the memory stores computer-executable instructions;
the processor executes computer-executable instructions stored in the memory to implement a training method for a credit risk scoring model as described in the first aspect, or to implement a credit risk scoring method as described in the second aspect.
In another aspect, the present application provides a computer-readable storage medium having stored therein computer-executable instructions that, when executed, cause a computer to perform a training method of a credit risk scoring model according to the first aspect, or to implement a credit risk scoring method according to the second aspect.
In another aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, implements a training method for a credit risk scoring model as described in the first aspect, or implements a credit risk scoring method as described in the second aspect.
The training method of the credit risk scoring model is used for training the risk scoring model which can carry out risk calculation on business data (including financial data and non-financial data) of an enterprise to obtain a risk value. Specifically, data of a plurality of financial indexes and data of a plurality of non-financial indexes of a plurality of enterprises are obtained, and a multi-dimensional time sequence data matrix is generated according to the data of the plurality of financial indexes and the data of the plurality of non-financial indexes. Obtaining an initial scoring model, wherein the initial scoring model comprises an ARIMA model and a back propagation BP neural network, the ARIMA model is used for outputting the prediction data of each data input to the ARIMA model in time, and the BP neural network is used for outputting the weight of each data input to the BP neural network and the weight of each prediction data. And taking data in the multi-dimensional time sequence data as training data, respectively training an ARIMA model and a BP neural network, and obtaining a target risk scoring model when a training termination condition is reached, wherein the target risk scoring model is used for generating a risk value of the data to be evaluated based on the input data to be evaluated.
The initial scoring model used in the training method of the credit risk scoring model provided by the embodiment of the application comprises an ARIMA model and a BP neural network, wherein the ARIMA model is a prediction model and is used for data prediction, and future business data of an enterprise can be reasonably inferred. The BP neural network is used for providing the weight required by calculating the risk value and providing a certain scientific basis for the calculation of the risk value. Therefore, when the trained target risk scoring model receives the data to be evaluated, the future business data of the enterprise can be reasonably speculated, and the risk value of the data to be evaluated can be obtained through calculation based on scientific weight. Therefore, the training method of the credit risk scoring model provided by the embodiment of the application can be used for objectively evaluating the credit risk of an enterprise based on real enterprise operation data, so that the accuracy of credit risk evaluation is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
Fig. 1 is a schematic view of an application scenario of a training method of a credit risk scoring model provided in the present application;
fig. 2 is a schematic flowchart of a training method of a credit risk scoring model according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a method for testing a target credit risk score model according to an embodiment of the present application;
fig. 4 is a schematic flowchart of a credit risk scoring method according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a training device for a credit risk scoring model according to an embodiment of the present application;
fig. 6 is a schematic diagram of a credit risk scoring device provided in an embodiment of the present application;
fig. 7 is a schematic diagram of an electronic device provided in an embodiment of the present application.
With the foregoing drawings in mind, certain embodiments of the disclosure have been shown and described in more detail below. These drawings and written description are not intended to limit the scope of the disclosed concepts in any way, but rather to illustrate the concepts of the disclosure to those skilled in the art by reference to specific embodiments.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
In the description of the present application, it is to be understood that the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or as implying that the number of indicated technical features is indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.
The terms referred to in this application are explained first:
credit risk: the credit risk is also called financial default risk, and refers to the default probability formed by the failure of the lender, the debtor or the transaction participant to fulfill the contract terms for various reasons, and the possibility of causing the loss of the transaction party and the investor. The root cause of credit risk is the severe information asymmetry present in the credit market, which leads to a reverse selection in credit risk.
ARIMA model: the ARIMA model is called as a differential autoregressive moving average model, and the basic idea is that a time sequence formed by a predicted object along with the time is regarded as a random sequence, the sequence is approximately described by a certain data model, and the model can predict future data through historical data and real-time data of the time sequence after being determined, and is generally applied to the field of finance.
BP neural network: the BP neural network is also called as a back propagation network, and through training of sample data, the weight and the threshold of the BP neural network are continuously corrected to enable an error function to descend along the direction of negative gradient and approach to expected output. The BP neural network is a wide neural network model and is mainly used for function approximation, model identification and classification, data compression, time series prediction and the like.
By 2020, the proportion of small and medium-sized enterprises registered in China to the whole number of registered industrial and commercial enterprises in China reaches 97%, and 65% of total domestic production value is provided. When a bank docks an enterprise, the credit risk of the enterprise is generally evaluated in order to avoid default. However, when a bank evaluates the credit risk of a small and medium enterprise, the bank often relies on the review experience of professional reviewers for evaluation, and the professional reviewers cannot make objective credit risk evaluation on the central enterprise due to the stereotyped impression of the small and medium enterprises (e.g., the enterprise has many operating conditions and short operating time). How to objectively evaluate the credit risk of an enterprise based on real enterprise operation data and improve the accuracy of credit risk evaluation still needs to be solved.
Based on the above, the application provides a training method of a credit risk scoring model and a credit risk scoring method. The training method of the credit risk scoring model is used for training a risk scoring model which can carry out risk calculation on business data (including financial data and non-financial data) of an enterprise to obtain a risk value. Specifically, data of a plurality of financial indexes and data of a plurality of non-financial indexes of a plurality of enterprises are obtained, and a multi-dimensional time sequence data matrix is generated according to the data of the plurality of financial indexes and the data of the plurality of non-financial indexes. An initial scoring model is obtained, the initial scoring model including an ARIMA model for outputting temporally predicted data for each data input to the ARIMA model and a back propagation BP neural network for outputting a weight for each data input to the BP neural network and a weight for each predicted data. And taking data in the multi-dimensional time sequence data as training data, respectively training the ARIMA model and the BP neural network, and obtaining a target risk scoring model when a training termination condition is reached, wherein the target risk scoring model is used for generating a risk value of the data to be evaluated based on the input data to be evaluated. Therefore, objective credit risk evaluation is performed on the enterprise through the real operation data of the enterprise by using the target risk scoring model, and the accuracy of the credit risk evaluation is improved.
The training method of the credit risk scoring model is applied to electronic equipment, such as a computer, a background server, a cloud server and the like. Fig. 1 is a schematic diagram of an application of the training method of the credit risk scoring model provided in the present application, in which the electronic device obtains an initial scoring model, and the initial scoring model includes a differential autoregressive moving average ARIMA model and a back propagation BP neural network. The method includes the steps of obtaining data of a plurality of financial indexes and data of a plurality of non-financial indexes of a plurality of enterprises, and generating a multi-dimensional time series data matrix according to the data of the plurality of financial indexes and the data of the plurality of non-financial indexes of the plurality of enterprises. And taking data in the multi-dimensional time sequence data as training data, respectively training the ARIMA model and the BP neural network, and obtaining a target risk scoring model when a training termination condition is reached.
Referring to fig. 2, an embodiment of the present application provides a trainer for a credit risk score model
A method, comprising:
s210, acquiring data of a plurality of financial indexes and data of a plurality of non-financial indexes of a plurality of enterprises, 5 the data of each financial index and the data of each non-financial index both carry time information, and obtaining the time information according to the financial indexes and the non-financial indexes
The data for the plurality of financial indicators and the data for the plurality of non-financial indicators generate a multi-dimensional time series data matrix.
The factors, characteristics and risk factor systems of the credit risk of the small and medium enterprises are systematically combed through related researches, the financial indexes cannot be simply relied on in the factors influencing the credit risk of the small and medium enterprises, and the non-financial indexes are comprehensively considered.
0 more financial indicators including, for example, four bases of asset liability, capital structure, revenue profit, and cash flow
The data index and four key grading indexes of repayment ability, profitability, operation ability and growth ability. Correspondingly, the data of the financial index is a specific numerical value. The data of the above eight financial indexes can be obtained through files such as annual newspapers of enterprises.
The non-financial indexes mainly include the business conditions of the enterprise, such as the business status, the management ability, the external environment, and the like, which are important 5, and mainly include the basic conditions of the enterprise (credit investigation records, litigation related information, and transparency of the enterprise), the basic conditions of the actual control personnel of the enterprise (personal credit investigation records, industry experience), and the like. The non-financial index has no specific numerical value, and the data of the non-financial index can be understood as the information of the counted non-financial index.
The data of each financial index and the data of each non-financial index carry time information, and the carried time information of 0 is the generation time of the data. When a multi-dimensional time sequence data matrix is generated according to data of a plurality of financial indexes and data of a plurality of non-financial indexes, the multi-dimensional time sequence data matrix is generated based on carried time information, wherein time sequence data of one dimension in the multi-dimensional time sequence data matrix is one-dimensional time sequence data formed by the data of one financial index or the data of a plurality of financial indexes.
Specifically, when a multi-dimensional time 5 time series data matrix is generated according to data of a plurality of financial indexes and data of a plurality of non-financial indexes, clustering processing is performed on the data of the plurality of non-financial indexes to obtain time series data of one dimension. And respectively taking the data of each financial index as time series data of one dimension to obtain time series data of multiple dimensions. Finally, time series data of one dimension based on data of a plurality of non-financial indexes and time of a plurality of dimensions based on data of a plurality of financial indexes
And (5) sequence data, and generating a multi-dimensional time sequence data matrix. As exemplified above for 8 financial indicators and 0 non-financial indicators, the multi-dimensional time series data matrix includes 9-dimensional time series data, 8-dimensional time series data in the 9-dimensional time series data correspond to the 8 financial indicators, and 1-dimensional time series data correspond to the non-financial indicators.
In an optional embodiment, when clustering is performed on data of a plurality of non-financial indexes to obtain time series data of one dimension, a preset classification standard is obtained first, where the preset classification standard includes M classes, and M is a natural number greater than 1. And respectively taking the randomly selected M pieces of data of the non-financial indexes as the first central point data of the M classes. At this time, the first central point data is the initial central point data selected randomly, and more accurate central point data needs to be determined again. The distances between the data according to each non-financial index and the first centre point data of the M classes, respectively, are then determined. And when the types of the data of the plurality of non-financial indexes are determined to be the first type, determining second central point data of the first type according to the data of the plurality of non-financial indexes. The determined second center point data is now more accurate. In order to determine fixed and unchangeable center point data, the step of repeatedly executing is that the data of M randomly selected non-financial indexes are respectively M types of first center point data, and the step of repeatedly executing is finished until the second center point data of the first type determined each time is a fixed value. And when the determined second center point data of the first class is a fixed value, determining the second center point data as time series data of one dimension.
In an alternative embodiment, the multi-dimensional time series data matrix is generated based on the time series data of one dimension obtained from the data of a plurality of non-financial indexes and the time series data of a plurality of dimensions obtained from the data of a plurality of financial indexes, and the method is divided into the following three steps.
The first step is as follows: and generating an initial multi-dimensional time sequence data matrix based on the time sequence data of one dimension obtained from the data of the plurality of non-financial indexes and the time sequence data of a plurality of dimensions obtained from the data of the plurality of financial indexes.
The initial multi-dimensional time series data matrix is X, which can be expressed as a matrix of m X n, which is
Figure BDA0003974141290000091
In the X matrix, X ij Representing the value of the data for the ith dimension at time j, each row in X is a time series.
The second step is that: and processing the initial multi-dimensional time series data matrix based on a sliding window algorithm with a preset acquisition step length to obtain a plurality of data sets.
Specifically, X is converted into a data set Z by a sliding window method, and if the step size of sliding is F + P, Z = { Z = { Z = 1 ,Z 2 ,...,Z L Where, L = n-F-P +1. Wherein Z is 1 、Z 2 、……、Z L Each representing a data set.
The third step: the multi-dimensional time series data matrix is generated based on the plurality of data sets.
According to Z = { Z 1 ,Z 2 ,...,Z L And constructing a multi-dimensional time sequence data matrix
Figure BDA0003974141290000101
Wherein j is more than or equal to 1 and less than or equal to n-F-P +1.
And S220, acquiring an initial scoring model, wherein the initial scoring model comprises a differential autoregressive moving average (ARIMA) model and a Back Propagation (BP) neural network, the ARIMA model is used for outputting prediction data of each data input to the ARIMA model in time, the BP neural network is used for outputting the weight of each data input to the BP neural network, and the data input to the BP neural network comprises the prediction data.
The ARIMA model has the basic idea that a time sequence formed by a prediction object along with the time is regarded as a random sequence, the sequence is approximately described by a certain data model, and the model can predict future data through historical data and real-time data of the time sequence after being determined, and is generally applied to the field of finance. Thus, the ARIMA model used by the present embodiment is used when applied to output temporally predictive data for each data input to the ARIMA model.
And the BP neural network continuously corrects the weight and the threshold value of the network through the training of sample data to enable the error function to descend along the direction of the negative gradient and approach to expected output. Therefore, the BP neural network used in the present embodiment is used to output a weight for each data input to the BP neural network when applied, and the data input to the BP neural network includes prediction data and training data for training the BP neural network. That is, in the initial scoring model, the output of the ARIMA model is input as part of the BP neural network.
And S230, taking data in the multi-dimensional time sequence data as training data, respectively training the ARIMA model and the BP neural network, and obtaining a target risk scoring model when a training termination condition is reached, wherein the target risk scoring model is used for generating a risk value of the data to be evaluated based on the input data to be evaluated.
In an alternative embodiment, the ARIMA model is trained with the first F column data in the multi-dimensional time series data as input data of the ARIMA model and the last P column data in the multi-dimensional time series data as reference data of the ARIMA model output data. As described above, the ARIMA model functions as data prediction, and in order to guarantee the training result of the ARIMA model, known data can be used as reference data to determine that the prediction data output by the ARIMA model is more accurate. And training the ARIMA model for multiple times until the training termination condition of the ARIMA model is reached to obtain the target ARIMA model.
The training termination conditions of the ARIMA model include one or more of the following: the difference value between the output data of the ARIMA model and the reference data of the output data of the ARIMA model is within a first preset range, the training time length reaches the preset time length, and the training times reaches the preset times. The first preset range may be set according to actual needs, and this embodiment is not limited.
Specifically, when the ARIMA model is trained using the first F-column data in the multidimensional time series data as input data of the ARIMA model, the ARIMA model is trained based on time series data of each dimension. For example, as described above, if there are 9 dimensions of time-series data, 9 ARIMA models are trained based on the 9 dimensions of time-series data, and an ARIMA model suitable for the time-series data of each dimension is obtained.
In an alternative embodiment, each datum with the same time in the multidimensional time series data is acquired as a first input datum of the BP neural network, a reference datum with known weight of the first input datum as an output datum of the BP neural network is acquired, and the output datum of the ARIMA model is taken as a second input datum of the BP neural network to train the BP neural network. As described above, the BP neural network is used to continuously modify the network weights and thresholds so that the error function falls along the negative gradient direction, approaching the desired output. The known weight of the first input data is reference data of the BP neural network output data, so that the weight output by the BP neural network is ensured to be more accurate. And training the BP neural network for multiple times until reaching the training termination condition of the BP neural network to obtain the target BP neural network.
The training termination condition of the BP neural network comprises one or more of the following conditions: and the difference value between the output data of the BP neural network and the reference data of the output data of the BP neural network is in a second preset range, the training time reaches the preset time, and the training times reaches the preset times. The second preset range may be set according to actual needs, and this embodiment is not limited.
And finally, after a target ARIMA model and a target BP neural network are obtained, obtaining a target risk scoring model based on the target ARIMA model and the target BP neural network, wherein the target risk scoring model is used for generating a risk value of the data to be evaluated according to the data to be evaluated, the predicted data of the data to be evaluated, the weight of the data to be evaluated and the weight of the predicted data of the data to be evaluated. For example, the data to be evaluated is a, the prediction data of the data to be evaluated is B, the weight of the data to be evaluated is a, and the weight of the prediction data of the data to be evaluated is B. Then, the risk value of the data to be evaluated = a + B.
Referring to fig. 3, in an alternative embodiment, the target risk scoring model may also be evaluated using known data. As shown in the figure, after data of a plurality of financial indexes and data of a plurality of non-financial indexes of a plurality of enterprises are collected, the data of the plurality of financial indexes and the data of the plurality of non-financial indexes are processed to obtain a multi-dimensional time sequence data matrix, and an ARIMA model and a BP neural network are trained by taking part of or all of the data in the multi-dimensional time sequence data matrix as a training set to obtain a target risk scoring model. And then, taking partial data of a plurality of financial indexes and non-financial indexes of a plurality of enterprises as a test set, or taking partial data in a multi-dimensional time sequence data matrix as the test set, and testing the target risk scoring model. The purpose of the test is to determine whether the risk value output by the target risk scoring model based on the test set substantially coincides with the known risk value of the test set, so as to learn the performance of the target risk scoring model.
In an alternative embodiment, the target risk scoring model is considered to be available for an actual risk scoring usage scenario when the risk value output by the target risk scoring model based on the test set substantially matches the known risk value of the test set (e.g., the difference is within a third predetermined range).
In an alternative embodiment, when the risk value output by the target risk scoring model based on the test set is different from the known risk value of the test set by a large amount (for example, the difference exceeds a third preset range), the training is continued to obtain a new target risk scoring model. And considering the new target risk scoring model to be used for the actual risk scoring use scene until the risk value output by the new target risk scoring model based on the test set is basically consistent with the known risk value of the test set.
In summary, the training method of the credit risk scoring model provided by the embodiment of the present application is used for training a risk scoring model that can perform risk calculation on business data (including financial data and non-financial data) of an enterprise to obtain a risk value. Specifically, data of a plurality of financial indexes and data of a plurality of non-financial indexes of a plurality of enterprises are obtained, and a multi-dimensional time sequence data matrix is generated according to the data of the plurality of financial indexes and the data of the plurality of non-financial indexes. An initial scoring model is obtained, the initial scoring model including an ARIMA model for outputting predicted data over time for each data input to the ARIMA model and a back propagation BP neural network for outputting a weight for each data input to the BP neural network and a weight for each predicted data. And taking data in the multi-dimensional time sequence data as training data, respectively training the ARIMA model and the BP neural network, and obtaining a target risk scoring model when a training termination condition is reached, wherein the target risk scoring model is used for generating a risk value of the data to be evaluated based on the input data to be evaluated.
The initial scoring model used in the training method of the credit risk scoring model provided by the embodiment of the application comprises an ARIMA model and a BP neural network, wherein the ARIMA model is a prediction model and is used for data prediction, and future business data of an enterprise can be reasonably inferred. The BP neural network is used for providing the weight required by calculating the risk value and providing a certain scientific basis for the calculation of the risk value. Therefore, when the trained target risk scoring model receives the data to be evaluated, the future business data of the enterprise can be reasonably speculated, and the risk value of the data to be evaluated can be obtained through calculation based on scientific weights. Therefore, the training method of the credit risk scoring model provided by the embodiment of the application can be used for objectively evaluating the credit risk of an enterprise based on real enterprise operation data, so that the accuracy of credit risk evaluation is improved.
Referring to fig. 4, an embodiment of the present application further provides a credit risk scoring method, including:
s410, obtaining data to be evaluated, wherein the data to be evaluated comprises data of a plurality of financial indexes and/or data of a plurality of non-financial indexes.
The data to be evaluated may further include data of other indexes, and when the data of the other indexes have numerical values, the data is used as time series data of one dimension, for example, data of a plurality of financial indexes is used as time series data of a plurality of dimensions. When the data of the other index does not have a numerical value, the data of a plurality of other indexes is used as the time-series data of one dimension, for example, the data of a plurality of non-financial indexes is used as the time-series data of one dimension.
And S420, inputting the data to be evaluated into a target risk scoring model obtained by training according to a preset method to obtain a risk value of the data to be evaluated.
The preset method is a training method of the credit risk scoring model provided in any one of the above embodiments.
The target risk scoring model is used for generating a risk value of the data to be evaluated based on the input data to be evaluated, and specifically, used for generating the risk value of the data to be evaluated according to the data to be evaluated, the prediction data of the data to be evaluated, the weight of the data to be evaluated and the weight of the prediction data of the data to be evaluated. And the predicted data of the data to be evaluated is output by the trained target ARIMA model, and the weight of the data to be evaluated and the weight of the predicted data of the data to be evaluated are output by the trained target BP neural network.
The initial scoring model used in the training method of the credit risk scoring model described above includes an ARIMA model and a BP neural network, wherein the ARIMA model is a prediction model used for data prediction, and future business data of an enterprise can be reasonably inferred. The BP neural network is used for providing the weight required by calculating the risk value and providing a certain scientific basis for the calculation of the risk value. Therefore, when the trained target risk scoring model receives the data to be evaluated, the future business data of the enterprise can be reasonably speculated, and the risk value of the data to be evaluated can be obtained through calculation based on scientific weight. Therefore, the training method of the credit risk scoring model provided by the embodiment of the application can be used for objectively evaluating the credit risk of an enterprise based on real enterprise operation data, so that the accuracy of credit risk evaluation is improved.
Referring to fig. 5, an embodiment of the present application further provides a training apparatus 10 for a credit risk scoring model, including:
the acquiring module 11 is configured to acquire data of a plurality of financial indexes and data of a plurality of non-financial indexes of a plurality of enterprises, where the data of each financial index and the data of each non-financial index both carry time information, and generate a multi-dimensional time series data matrix according to the data of the plurality of financial indexes and the data of the plurality of non-financial indexes.
The obtaining module 11 is further configured to obtain an initial scoring model, where the initial scoring model includes a differential autoregressive moving average ARIMA model and a back propagation BP neural network, the ARIMA model is configured to output predicted data of each data input to the ARIMA model over time, and the BP neural network is configured to output a weight of each data input to the BP neural network, where the data input to the BP neural network includes the predicted data.
The training module 12 is configured to use data in the multi-dimensional time series data as training data, respectively train the ARIMA model and the BP neural network, and obtain a target risk score model when a training termination condition is reached, where the target risk score model is used to generate a risk value of the data to be evaluated based on the input data to be evaluated.
The training module 12 is specifically configured to train the ARIMA model by using the first F column data in the multidimensional time series data as input data of the ARIMA model and using the last P column data in the multidimensional time series data as reference data of output data of the ARIMA model, until a training termination condition of the ARIMA model is reached, obtaining a target ARIMA model, where the training termination condition of the ARIMA model includes one or more of: the difference value between the output data of the ARIMA model and the reference data of the output data of the ARIMA model is within a first preset range, the training time length reaches the preset time length, and the training times reaches the preset times; acquiring first input data of a BP (back propagation) neural network, which are data with the same time in the multi-dimensional time sequence data, acquiring reference data of which the known weight of the first input data is output data of the BP neural network, and training the BP neural network by using the output data of the ARIMA model as second input data of the BP neural network until a training termination condition of the BP neural network is reached to obtain a target BP neural network, wherein the training termination condition of the BP neural network comprises one or more of the following conditions: the difference value between the output data of the BP neural network and the reference data of the output data of the BP neural network is in a second preset range, the training time length reaches the preset time length, and the training times reaches the preset times; and obtaining a target risk scoring model based on the target ARIMA model and the target BP neural network, wherein the target risk scoring model is used for generating a risk value of the data to be evaluated according to the data to be evaluated, the predicted data of the data to be evaluated, the weight of the data to be evaluated and the weight of the predicted data of the data to be evaluated.
The obtaining module 11 is specifically configured to perform clustering processing on data of a plurality of non-financial indexes to obtain time series data of one dimension; respectively taking the data of each financial index as time series data of one dimension to obtain time series data of multiple dimensions; and generating a multi-dimensional time series data matrix based on the time series data of one dimension obtained from the data of the plurality of non-financial indexes and the time series data of a plurality of dimensions obtained from the data of the plurality of financial indexes.
The obtaining module 11 is specifically configured to generate an initial multi-dimensional time series data matrix based on time series data of one dimension obtained from data of a plurality of non-financial indexes and time series data of a plurality of dimensions obtained from data of a plurality of financial indexes; processing the initial multi-dimensional time series data matrix based on a sliding window algorithm with a preset acquisition step length to obtain a plurality of data sets; the multi-dimensional time series data matrix is generated based on the plurality of data sets.
The obtaining module 11 is specifically configured to obtain a preset classification standard, where the preset classification standard includes M classes, and M is a natural number greater than 1; respectively taking the randomly selected M pieces of data of the non-financial indexes as M pieces of first central point data; determining second central point data of a first type according to the data of the plurality of non-financial indexes when the types of the data of the plurality of non-financial indexes are determined to be the first type according to the distance between the data of each non-financial index and the first central point data of the M types; and repeating the step, wherein the randomly selected M pieces of data of the non-financial indexes are respectively M pieces of first central point data, and the second central point data is determined to be time sequence data of one dimension when the second central point data of the first type is determined to be a fixed value every time.
Referring to fig. 6, an embodiment of the present application further provides a credit risk scoring apparatus 20, including:
the obtaining module 21 is configured to obtain data to be evaluated, where the data to be evaluated includes data of multiple financial indexes and/or data of multiple non-financial indexes;
and the evaluation module 22 is configured to input the data to be evaluated to a target risk scoring model obtained by training in a preset method, so as to obtain a risk value of the data to be evaluated.
Referring to fig. 7, an embodiment of the present application further provides an electronic device 30, including: a processor 31, and a memory 32 communicatively coupled to the processor 31. The memory 32 stores computer-executable instructions, and the processor 31 executes the computer-executable instructions stored by the memory 32 to implement a training method of a credit risk scoring model as provided in any of the above embodiments, or to implement a credit risk scoring method as provided in any of the above embodiments.
The present application also provides a computer-readable storage medium having stored therein computer-executable instructions that, when executed, cause a computer to execute the instructions for implementing a training method for a credit risk scoring model as provided in any of the above embodiments, or implementing a credit risk scoring method as provided in any of the above embodiments.
The present application further provides a computer program product comprising a computer program which, when executed by a processor, implements a training method for a credit risk scoring model as provided in any of the above embodiments, or implements a credit risk scoring method as provided in any of the above embodiments.
The computer-readable storage medium may be a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read Only Memory (EPROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a magnetic Random Access Memory (FRAM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical Disc, or a Compact Disc Read-Only Memory (CD-ROM). And may be various electronic devices such as mobile phones, computers, tablet devices, personal digital assistants, etc., including one or any combination of the above-mentioned memories.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a component of' 8230; \8230;" does not exclude the presence of another like element in a process, method, article, or apparatus that comprises the element.
The above-mentioned serial numbers of the embodiments of the present application are merely for description, and do not represent the advantages and disadvantages of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method described in the embodiments of the present application.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present application, and not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application, or which are directly or indirectly applied to other related technical fields, are included in the scope of the present application.

Claims (10)

1. A training method of a credit risk scoring model is characterized by comprising the following steps:
acquiring data of a plurality of financial indexes and data of a plurality of non-financial indexes of a plurality of enterprises, wherein the data of each financial index and the data of each non-financial index carry time information, and generating a multi-dimensional time sequence data matrix according to the data of the plurality of financial indexes and the data of the plurality of non-financial indexes;
acquiring an initial scoring model, wherein the initial scoring model comprises a differential autoregressive moving average (ARIMA) model and a Back Propagation (BP) neural network, the ARIMA model is used for outputting prediction data of each data input to the ARIMA model in time, the BP neural network is used for outputting the weight of each data input to the BP neural network, and the data input to the BP neural network comprises the prediction data;
and taking data in the multi-dimensional time sequence data as training data, respectively training an ARIMA model and a BP neural network, and obtaining a target risk scoring model when a training termination condition is reached, wherein the target risk scoring model is used for generating a risk value of the data to be evaluated based on the input data to be evaluated.
2. The method as claimed in claim 1, wherein the training the ARIMA model and the BP neural network respectively with data in the multidimensional time series data as training data to obtain the target risk score model comprises:
and taking front F column data in the multi-dimensional time sequence data as input data of an ARIMA model, taking back P column data in the multi-dimensional time sequence data as reference data of output data of the ARIMA model, training the ARIMA model until a training termination condition of the ARIMA model is reached to obtain a target ARIMA model, wherein the training termination condition of the ARIMA model comprises one or more of the following conditions: the difference value between the output data of the ARIMA model and the reference data of the output data of the ARIMA model is within a first preset range, the training time length reaches the preset time length, and the training times reaches the preset times;
acquiring first input data of a BP (back propagation) neural network, which are data with the same time in the multi-dimensional time sequence data, acquiring reference data of which the known weight of the first input data is output data of the BP neural network, and training the BP neural network by using the output data of the ARIMA model as second input data of the BP neural network until a training termination condition of the BP neural network is reached to obtain a target BP neural network, wherein the training termination condition of the BP neural network comprises one or more of the following conditions: the difference value between the output data of the BP neural network and the reference data of the output data of the BP neural network is in a second preset range, the training time length reaches the preset time length, and the training times reaches the preset times;
and obtaining a target risk scoring model based on the target ARIMA model and the target BP neural network, wherein the target risk scoring model is used for generating a risk value of the data to be evaluated according to the data to be evaluated, the predicted data of the data to be evaluated, the weight of the data to be evaluated and the weight of the predicted data of the data to be evaluated.
3. The method of claim 1 or 2, wherein generating the multi-dimensional time series data matrix from the data for the plurality of financial indicators and the data for the plurality of non-financial indicators comprises:
clustering data of a plurality of non-financial indexes to obtain time sequence data of one dimension;
respectively taking the data of each financial index as time series data of one dimension to obtain time series data of multiple dimensions;
and generating a multi-dimensional time series data matrix based on the time series data of one dimension obtained from the data of the plurality of non-financial indexes and the time series data of a plurality of dimensions obtained from the data of the plurality of financial indexes.
4. The method of claim 3, wherein generating a multi-dimensional time series data matrix based on the time series data for the one dimension derived from the data for the plurality of non-financial indicators and the time series data for the plurality of dimensions derived from the data for the plurality of financial indicators comprises:
generating an initial multi-dimensional time series data matrix based on time series data of one dimension obtained from data of a plurality of non-financial indexes and time series data of a plurality of dimensions obtained from data of a plurality of financial indexes;
processing the initial multi-dimensional time series data matrix based on a sliding window algorithm with a preset acquisition step length to obtain a plurality of data sets;
generating the multi-dimensional time series data matrix based on the plurality of data sets.
5. The method of claim 3, wherein clustering data of a plurality of non-financial indicators to obtain time series data of a dimension comprises:
acquiring a preset classification standard, wherein the preset classification standard comprises M classes, and M is a natural number greater than 1;
respectively taking the randomly selected M pieces of data of the non-financial indexes as M pieces of first central point data;
determining second central point data of a first type according to the data of the plurality of non-financial indexes when the types of the data of the plurality of non-financial indexes are determined to be the first type according to the distance between the data of each non-financial index and the first central point data of the M types;
and repeating the step, wherein the randomly selected M pieces of data of the non-financial indexes are respectively M pieces of first central point data, and the second central point data is determined to be time sequence data of one dimension when the second central point data of the first type is determined to be a fixed value every time.
6. A credit risk scoring method, comprising:
acquiring data to be evaluated, wherein the data to be evaluated comprises data of a plurality of financial indexes and/or data of a plurality of non-financial indexes;
inputting the data to be evaluated into a target risk scoring model obtained by training the training method of the credit risk scoring model according to any one of claims 1 to 5, and obtaining the risk value of the data to be evaluated.
7. A training device for a credit risk scoring model, comprising:
the system comprises an acquisition module, a time information acquisition module and a time information acquisition module, wherein the acquisition module is used for acquiring data of a plurality of financial indexes and data of a plurality of non-financial indexes of a plurality of enterprises, and generating a multi-dimensional time sequence data matrix according to the data of the financial indexes and the data of the non-financial indexes;
the obtaining module is further configured to obtain an initial scoring model, where the initial scoring model includes a differential autoregressive moving average (ARIMA) model and a Back Propagation (BP) neural network, the ARIMA model is configured to output predicted data of each data input to the ARIMA model over time, the BP neural network is configured to output weights of each data input to the BP neural network, and the data input to the BP neural network includes the predicted data;
and the training module is used for respectively training an ARIMA model and a BP neural network by taking data in the multi-dimensional time sequence data as training data to obtain a target risk scoring model when a training termination condition is reached, and the target risk scoring model is used for generating a risk value of the data to be evaluated based on the input data to be evaluated.
8. A credit risk scoring apparatus, comprising:
the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring data to be evaluated, and the data to be evaluated comprises data of a plurality of financial indexes and/or data of a plurality of non-financial indexes;
the evaluation module is used for inputting the data to be evaluated into the target risk scoring model obtained by training the training method of the credit risk scoring model according to any one of claims 1 to 5, so as to obtain the risk value of the data to be evaluated.
9. An electronic device, comprising: a processor, and a memory communicatively coupled to the processor;
the memory stores computer-executable instructions;
the processor executes the computer-executable instructions stored in the memory to implement the training method of the credit risk scoring model of any one of claims 1 to 5 or to implement the credit risk scoring method of claim 6.
10. A computer-readable storage medium having computer-executable instructions stored therein, which when executed, cause a computer to perform the training method of the credit risk scoring model of any one of claims 1 to 5 or to implement the credit risk scoring method of claim 6.
CN202211521953.3A 2022-11-30 2022-11-30 Training method of credit risk scoring model and credit risk scoring method Pending CN115829722A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211521953.3A CN115829722A (en) 2022-11-30 2022-11-30 Training method of credit risk scoring model and credit risk scoring method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211521953.3A CN115829722A (en) 2022-11-30 2022-11-30 Training method of credit risk scoring model and credit risk scoring method

Publications (1)

Publication Number Publication Date
CN115829722A true CN115829722A (en) 2023-03-21

Family

ID=85533184

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211521953.3A Pending CN115829722A (en) 2022-11-30 2022-11-30 Training method of credit risk scoring model and credit risk scoring method

Country Status (1)

Country Link
CN (1) CN115829722A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116258579A (en) * 2023-04-28 2023-06-13 成都新希望金融信息有限公司 Training method of user credit scoring model and user credit scoring method

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116258579A (en) * 2023-04-28 2023-06-13 成都新希望金融信息有限公司 Training method of user credit scoring model and user credit scoring method

Similar Documents

Publication Publication Date Title
US10482079B2 (en) Data de-duplication systems and methods
US11599939B2 (en) System, method and computer program for underwriting and processing of loans using machine learning
US20180260891A1 (en) Systems and methods for generating and using optimized ensemble models
US8489502B2 (en) Methods and systems for multi-credit reporting agency data modeling
CN108475393A (en) The system and method that decision tree is predicted are promoted by composite character and gradient
CN104321794B (en) A kind of system and method that the following commercial viability of an entity is determined using multidimensional grading
Kuzior et al. Global digital convergence: Impact of cybersecurity, business transparency, economic transformation, and AML efficiency
CN111882140A (en) Risk evaluation method, model training method, device, equipment and storage medium
CN115829722A (en) Training method of credit risk scoring model and credit risk scoring method
CN110796539A (en) Credit investigation evaluation method and device
CN112434862B (en) Method and device for predicting financial dilemma of marketing enterprises
CN114493142A (en) Method, device, equipment and storage medium for matching support policy with enterprise
CN117252677A (en) Credit line determination method and device, electronic equipment and storage medium
CN113706258A (en) Product recommendation method, device, equipment and storage medium based on combined model
CN111160929B (en) Method and device for determining client type
CN113177733A (en) Medium and small micro-enterprise data modeling method and system based on convolutional neural network
CN112884301A (en) Method, equipment and computer storage medium for enterprise risk analysis
Tselekidou A Machine Learning Approach for Micro-Credit Scoring and Limit Optimization
CN116644372B (en) Account type determining method and device, electronic equipment and storage medium
CN113656692B (en) Product recommendation method, device, equipment and medium based on knowledge migration algorithm
Anusha et al. An Approach to Loan Approval prediction Using Boosting Ensemble Learning
Bøe Predicting defaults in the automotive credit Industry: an empircial study using machine learning techniques predicting loan defaults
CN117710095A (en) Risk assessment method, device, equipment and storage medium based on assessment model
CN117196827A (en) Method, device, equipment and storage medium for predicting early compensation rate of house loan
CN114997999A (en) Artificial intelligence financial data management system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination