CN113887195A - Contract review method, device, equipment and storage medium based on artificial intelligence - Google Patents
Contract review method, device, equipment and storage medium based on artificial intelligence Download PDFInfo
- Publication number
- CN113887195A CN113887195A CN202111152131.8A CN202111152131A CN113887195A CN 113887195 A CN113887195 A CN 113887195A CN 202111152131 A CN202111152131 A CN 202111152131A CN 113887195 A CN113887195 A CN 113887195A
- Authority
- CN
- China
- Prior art keywords
- contract
- matrix
- syntactic analysis
- transformer model
- clauses
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Probability & Statistics with Applications (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Machine Translation (AREA)
Abstract
The application belongs to the technical field of artificial intelligence and provides a contract review method, a contract review device, contract review equipment and a storage medium based on artificial intelligence, wherein the method comprises the following steps: obtaining a sample contract, and carrying out syntactic analysis on the clauses of the sample contract to obtain a syntactic analysis binary tree corresponding to the clauses of the sample contract; training the improved Transformer model with the enhanced attention mechanism according to the clause of the sample contract and the syntactic analysis binary tree corresponding to the clause to obtain a trained improved Transformer model; acquiring a contract to be evaluated, and predicting a target sentence embedding expression matrix corresponding to the clause of the contract to be evaluated by using a trained improved Transformer model; and classifying the target sentence embedding expression matrix by using a pre-trained classification model, and obtaining an evaluation result of whether the contract to be evaluated lacks terms according to the classification result. The method and the device can improve the training speed and effect of the model and improve the contract review accuracy.
Description
Technical Field
The application relates to the technical field of artificial intelligence, in particular to a contract review method, a contract review device, contract review equipment and a storage medium based on artificial intelligence.
Background
At present, the missing review of the necessary terms of the same is an indispensable important link in the legal review. With the gradual improvement of the law, the types and the quantity of the terms are large, and the traditional mode of relying on manual review undoubtedly consumes huge manpower.
In the related art, a transform-based language model is used to evaluate the agreement, such as bert, xlnet, etc. When the language model is trained, the attention mechanism of the transformer is to consider the weight distribution of a single character on the whole sentence, contract terms are typical long texts, the attention of the single character on the long texts is too sparsely dispersed, the model training difficulty is increased, the model precision is low, the sparse dispersion of the attention is bound to generate too large supermatrix operation, the model training speed is reduced, and the problems of too long training time and low evaluation accuracy of the model in contract evaluation application exist.
Disclosure of Invention
The application mainly aims to provide a contract review method, a contract review device, a contract review equipment and a storage medium based on artificial intelligence, and aims to solve the technical problems that in the related technology, a language model based on a Transformer is adopted to review contracts, the model training time is too long, and the review accuracy is low.
In a first aspect, the present application provides a contract review method based on artificial intelligence, the method comprising:
obtaining a sample contract, and carrying out syntactic analysis on clauses of the sample contract to obtain a syntactic analysis binary tree corresponding to the clauses of the sample contract;
training the improved Transformer model with the enhanced attention mechanism according to the clause of the sample contract and the syntactic analysis binary tree corresponding to the clause to obtain a trained improved Transformer model;
acquiring a contract to be evaluated and examined, and predicting a target sentence embedding expression matrix corresponding to the clause of the contract to be evaluated and examined by using the trained improved Transformer model;
and classifying the target sentence embedding expression matrix by utilizing a pre-trained classification model, and obtaining an evaluation result of whether the contract to be evaluated lacks terms according to the classification result.
In a second aspect, the present application further provides an artificial intelligence based contract review device, which includes:
the analysis module is used for obtaining a sample contract, and performing syntactic analysis on clauses of the sample contract to obtain a syntactic analysis binary tree corresponding to the clauses of the sample contract;
the training module is used for training the improved Transformer model with the enhanced attention mechanism according to the clauses of the sample contract and the syntactic analysis binary tree corresponding to the clauses to obtain a trained improved Transformer model;
the prediction module is used for acquiring a contract to be evaluated and examined and predicting a target sentence embedding expression matrix corresponding to the clause of the contract to be evaluated and examined by using the trained improved Transformer model;
and the classification module is used for classifying the target sentence embedding expression matrix by using a pre-trained classification model and obtaining an evaluation result of whether the contract to be evaluated lacks terms according to the classification result. In a third aspect, the present application also provides a computer device comprising a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein the computer program, when executed by the processor, implements the steps of the artificial intelligence based contract review method as described above.
In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the artificial intelligence based contract review method as described above.
The contract review method based on the artificial intelligence obtains a sample contract, and carries out syntactic analysis on the clauses of the sample contract to obtain a syntactic analysis binary tree corresponding to the clauses of the sample contract; training the improved Transformer model with the enhanced attention mechanism according to the clause of the sample contract and the syntactic analysis binary tree corresponding to the clause to obtain a trained improved Transformer model; acquiring a contract to be evaluated and examined, and predicting a target sentence embedding expression matrix corresponding to the clause of the contract to be evaluated and examined by using the trained improved Transformer model; and classifying the target sentence embedding expression matrix by utilizing a pre-trained classification model, and obtaining an evaluation result of whether the contract to be evaluated lacks terms according to the classification result. Because the improved Transformer model enhances the original attention mechanism of the Transformer model, the attention distribution of the improved Transformer model is not dispersed, the training speed and the model precision can be effectively improved during training, the trained improved Transformer model can more quickly obtain a sentence embedding expression matrix capable of accurately representing the clause semantics of the contract to be reviewed, and the contract review is realized through the classification model on the basis of the sentence embedding expression matrix of the clause of the contract to be reviewed, so that the review efficiency and the review accuracy are greatly improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic flow chart diagram illustrating an embodiment of a contract review method based on artificial intelligence according to the present application;
FIG. 2 is a flowchart of a calculation of an attention mechanism after enhancing a Transformer model according to an embodiment of the artificial intelligence based contract review method;
FIG. 3 is a schematic block diagram of an artificial intelligence based contract review apparatus according to an embodiment of the present application;
fig. 4 is a block diagram illustrating a structure of a computer device according to an embodiment of the present application.
The implementation, functional features and advantages of the objectives of the present application will be further explained with reference to the accompanying drawings.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The flow diagrams depicted in the figures are merely illustrative and do not necessarily include all of the elements and operations/steps, nor do they necessarily have to be performed in the order depicted. For example, some operations/steps may be decomposed, combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
It is to be understood that the terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification of the present application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
The embodiment of the application provides a contract review method, a contract review device, contract review equipment and a storage medium based on artificial intelligence. The artificial intelligence based contract review method is mainly applied to artificial intelligence based contract review equipment, and the artificial intelligence based contract review equipment can be terminal equipment with a data processing function, such as a server and the like.
The server may be an independent server, or may be a cloud server that provides basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a Content Delivery Network (CDN), a big data and artificial intelligence platform, and the like.
Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the features of the embodiments can be combined with each other without conflict.
Referring to fig. 1, fig. 1 is a schematic flow chart of a contract review method based on artificial intelligence according to an embodiment of the present application.
As shown in FIG. 1, the artificial intelligence based contract review method includes steps S101 to S104.
Step S101, a sample contract is obtained, and the clauses of the sample contract are subjected to syntactic analysis to obtain a syntactic analysis binary tree corresponding to the clauses of the sample contract.
The contract review method based on artificial intelligence mainly comprises two processes, namely training an improved transformer model and a classification model to obtain the trained improved transformer model and the trained classification model, wherein the classification model can be a softmax classifier; and secondly, adopting the trained improved transformer model and classification model to realize the review of the contract to be reviewed.
In order to improve the evaluation accuracy, the method is different from a transform model in the related technology, the original Attention (Attention) mechanism of the transform model is creatively enhanced, the enhanced Attention mechanism can effectively gather the Attention of text characters and aim at training, the calculated amount of the transform model is greatly reduced, and the training speed and the prediction accuracy of the model can be improved.
the Transformer model is an NLP (natural language processing) model, the Transformer model completely depends on an Attention mechanism to calculate the input and output characteristics of the model, and a cyclic neural network and a convolutional neural network adopted by the traditional NLP model are abandoned, so that the Transformer model greatly relieves the problems of gradient disappearance and gradient explosion. The Transformer model mainly comprises an Encoder and a Decoder, wherein the Encoder and the Decoder both comprise the functions of Attention generation, forward propagation and the like.
Firstly, introducing an original attention mechanism of a Transformer model, wherein the original attention mechanism calculation process of the Transformer model mainly comprises the following two steps:
1) when a sentence is input into a Transformer model, a word vector matrix X of the input sentence is subjected to linear transformation to generate a query (Q, queries) matrix, a key (K, keys) matrix and a value (V, values) matrix;
2) q, K the two matrixes are multiplied to obtain an attention matrix, and then the attention matrix is multiplied by the V matrix to obtain an embedded expression matrix of the sentence.
It can be seen from this that the original attention mechanism of the transform model is to consider the weight distribution of a single character over the whole sentence, and there are two disadvantages: 1) for long sentences, the attention of a single character on the sentence is sparsely dispersed, so that the difficulty of model training is increased, and the precision of the model training is low; 2) the sparse dispersion of attention will inevitably produce too large a supermtrix operation, making the model training speed slow.
Aiming at the defects of the original attention mechanism of the Transformer model, the original attention mechanism of the Transformer model is improved to obtain an improved Transformer model with enhanced attention mechanism, please refer to fig. 2, and fig. 2 is a calculation flow of the attention mechanism after the improved Transformer model is enhanced, and the calculation flow comprises the following steps:
1) constructing a syntax analysis masking matrix:
a. the syntax analysis tool of Stanford Parser is used to analyze the syntax of the sentence, the syntax analysis result is presented in the form of binary tree, the leaf node corresponds to each character in the sentence (one-to-one), and the distance dis (i, j) between any two characters is defined as the distance between two nodes on the corresponding binary tree.
b. And if the sentence length is l, constructing a syntactic analysis mask matrix M with the size of (l, l), and defining a threshold value M, wherein when dis (i, j) > M and M [ i, j ] are equal to negative infinity, and when dis (i, j) < ═ M, M [ i, j ] ═ 0.
2) Generating a new attention matrix:
a. multiplying the Q matrix and the K matrix of the Transformer to obtain a first attention matrix A with the size of (l, l), and adding the attention matrix A and the syntax analysis and masking matrix M to obtain a second new attention matrix G with the size of (l, l). It can be understood that when M [ i, j ] is equal to negative infinity, then G [ i, j ] is equal to negative infinity, which means that i and j are far enough apart, attention is not required, and in the model training back propagation, the gradient at negative infinity is 0, training is not required, and the model training speed can be greatly accelerated; when M [ i, j ] is equal to 0, G [ i, j ] is equal to A [ i, j ], the original attention weight is maintained.
b. Setting a gate mechanism alpha, and blending A, G two attention matrix matrixes, wherein the final enhanced new attention matrix is alpha A + (1-alpha) G.
3) Generating a new sentence embedding representation matrix:
the final new sentence embedding matrix represents the result of multiplication of matrices equal to alpha A + (1-alpha) G and V.
In order to apply the attention mechanism enhanced improved transformer model to contract review, the attention mechanism enhanced improved transformer model is trained according to the sample contract to obtain a trained improved transformer model, and the following is a training process:
a sample contract is first obtained, the sample contract including a number of terms. Before a contract sample is adopted to train an improved Transformer model, firstly, paragraph splitting is carried out on the sample contract to obtain clauses of the sample contract, then, a Stanford syntactic analysis tool is adopted to carry out syntactic analysis on the sample contract to obtain a syntactic analysis binary tree corresponding to the clauses of the sample contract, wherein leaf nodes of the syntactic analysis binary tree correspond to each character in the clauses of the sample contract.
And S102, training the improved Transformer model with the enhanced attention mechanism according to the clause of the sample contract and the syntactic analysis binary tree corresponding to the clause of the sample contract to obtain the trained improved Transformer model.
And then, training the improved Transformer model according to the clauses of the sample contract and the syntactic analysis binary tree corresponding to the clauses of the sample contract to obtain the trained improved Transformer model.
In some embodiments, the training of the improved Transformer model with enhanced attention mechanism according to the terms of the sample contract and the syntactic analysis binary tree corresponding to the terms of the sample contract to obtain the trained improved Transformer model specifically includes: inputting the clauses of the sample contract and the syntactic analysis binary tree corresponding to the clauses into an improved Transformer model, so that the attention mechanism enhanced by the improved Transformer model generates a query matrix, a key matrix and a value matrix according to the clauses of the sample contract, and constructs a syntactic analysis mask matrix according to the syntactic analysis binary tree; and training the improved Transformer model according to the query matrix, the key matrix, the value matrix and the syntactic analysis masking matrix to obtain the trained improved Transformer model.
The method comprises the steps of taking clauses of a sample contract and a syntactic analysis binary tree corresponding to the clauses of the sample contract as input of an improved Transformer model, improving an attention mechanism of the enhanced Transformer model, firstly generating a query matrix, a key matrix and a value matrix based on the clauses of the sample contract, constructing a syntactic analysis mask matrix based on the syntactic analysis binary tree corresponding to the clauses of the sample contract, and then training the improved Transformer model after the attention mechanism is enhanced according to the query matrix, the key matrix and the value matrix corresponding to the clauses of the sample contract and the syntactic analysis mask matrix to obtain the trained improved Transformer model.
In some embodiments, the training of the improved Transformer model according to the query matrix, the key matrix, the value matrix, and the syntactic analysis masking matrix is performed to obtain a trained improved Transformer model, which specifically includes: multiplying the query matrix and the key matrix to obtain a first attention matrix; adding the attention matrix and the syntactic analysis covering matrix to obtain a second attention matrix; blending the first attention matrix and the second attention matrix to obtain an enhanced new attention matrix; and multiplying the enhanced new attention matrix and the value matrix to obtain a sentence embedding expression matrix corresponding to the clause of the sample contract.
The method comprises the steps of multiplying a query matrix and a key matrix corresponding to the clause of a sample contract to obtain a first attention matrix, adding the first attention matrix and a syntactic analysis masking matrix corresponding to the clause of the sample contract to obtain a second attention matrix, harmonizing the first attention matrix and the second attention matrix to obtain an enhanced new attention matrix, multiplying the enhanced new attention matrix and a value matrix corresponding to the clause of the sample contract to obtain a sentence embedding expression matrix corresponding to the clause of the sample contract, adjusting parameters of an improved Transformer model according to the sentence embedding expression matrix corresponding to the clause of the sample contract until the Transformer model converges, and obtaining the trained improved Transformer model.
In some embodiments, the blending the first attention matrix and the second attention matrix to obtain an enhanced new attention matrix specifically includes: acquiring a preset blending formula alpha A + (1-alpha) G, wherein alpha represents a preset door mechanism, A represents the first attention matrix, and G represents the second new attention matrix; and substituting the first attention matrix and the second attention matrix into the preset harmonic formula for calculation to obtain an enhanced new attention matrix.
Namely, substituting the first attention matrix and the second attention matrix corresponding to the clause of the sample contract into a preset blending formula shown as follows to calculate, so as to obtain an enhanced new attention matrix corresponding to the clause of the sample contract:
alpha*A+(1-alpha)*G
where alpha represents a preset gate mechanism, A represents a first attention matrix, and G represents a second new attention matrix.
The improved Transformer model is trained by adopting the sample contract, and the attention mechanism enhanced by the improved Transformer model can effectively gather the attention of the characters of the clauses of the sample contract and aim at training, so that the training speed and the training precision can be improved, and the trained improved Transformer model has higher accuracy in contract review application.
Further, after the training of the improved Transformer model is completed, the sentence corresponding to the clause of the sample contract is embedded into the representation matrix and the real category of the clause of the sample contract, and the representation matrix and the real category of the clause of the sample contract are used as samples for training the softmax classifier, and the softmax classifier is trained, wherein the real category of the clause of the sample contract can be obtained by pre-labeling.
The softmax classifier is a common linear classifier and is suitable for multi-classification prediction problems. Specifically, the sentence embedding expression matrix corresponding to the clause of the sample contract and the real category of the clause of the sample contract are used as input of the softmax classifier, probability predicted values of the sentence embedding expression matrix corresponding to the clause of the sample contract belonging to all preset categories are obtained, the probability predicted values of the sentence embedding expression matrix belonging to all preset standard clause categories and the real category of the clause of the sample contract are compared, cross soil moisture loss is established to obtain a loss function of the softmax classifier, the loss function is optimized by a gradient descent method, the loss function is made to be smaller and smaller until the loss function is converged, and the trained softmax classifier can be obtained. The input of the trained softmax classifier is sentence embedding expression matrix, and the output is classification probability value between [0, 1 ]. The trained softmax classifier can realize accurate classification of the same clauses.
Step S103, acquiring a contract to be evaluated and examined, and predicting a target sentence embedding expression matrix corresponding to the clause of the contract to be evaluated and examined by using the trained improved Transformer model.
The process of reviewing the to-be-reviewed contract using the trained improved Transformer model and the softmax classifier is as follows.
Firstly, acquiring a contract to be reviewed, and predicting a target sentence embedding expression matrix corresponding to the clause of the contract to be reviewed by using a trained improved Transformer model.
In some embodiments, the method for predicting the target sentence embedding expression matrix corresponding to the clause of the contract to be reviewed by using the trained improved Transformer model specifically includes: performing syntactic analysis on the terms of the contract to be evaluated to obtain a target syntactic analysis binary tree corresponding to the terms of the contract to be evaluated; and inputting the clauses of the contract to be evaluated and examined and the target syntactic analysis binary tree corresponding to the clauses into the trained improved Transformer model for prediction to obtain the target sentence embedding expression matrix.
The syntactic analysis is performed on the clauses of the contract to be reviewed to obtain a target syntactic analysis binary tree corresponding to the clauses of the contract to be reviewed, and the method specifically comprises the following steps: carrying out paragraph splitting on the contract to be evaluated to obtain the clause of the contract to be evaluated; and carrying out syntactic analysis on the clauses of the contract to be reviewed by using a Stanford syntactic analysis tool to obtain the target syntactic analysis binary tree.
Similarly, the paragraph splitting is performed on the contract to be evaluated to obtain the clause of the contract to be evaluated, and then the syntax analysis is performed on the clause of the contract to be evaluated by using the Stanford syntax analysis tool to obtain the target syntax analysis binary tree corresponding to the clause of the contract to be evaluated, wherein the leaf node of the target syntax analysis binary tree corresponds to each character of the clause of the contract to be evaluated.
Inputting the clauses of the contract to be reviewed and the corresponding target syntactic analysis binary tree into a trained improved Transformer model, wherein the trained improved Transformer model firstly generates a query matrix, a key matrix and a value matrix based on the clauses of the contract to be reviewed and constructs a target syntactic analysis masking matrix based on the target syntactic analysis binary tree corresponding to the clauses of the contract to be reviewed through an enhanced attention mechanism, multiplies the query matrix and the key matrix corresponding to the clauses of the contract to obtain a first target attention matrix, adds the first target attention matrix and the target syntactic analysis masking matrix to obtain a second target attention matrix, harmonizes the first target attention matrix and the second target attention matrix to obtain an enhanced new target attention matrix, multiplies the enhanced new target attention matrix and the value matrix corresponding to the clauses of the contract to be reviewed, and obtaining and outputting a target sentence embedding expression matrix corresponding to the clause of the contract to be evaluated and examined.
The sentence embedding expression matrix capable of accurately representing the clause semantics of the contract to be evaluated can be obtained more quickly through the trained improved Transformer model.
And step S104, classifying the target sentence embedding expression matrix by utilizing a pre-trained classification model, and obtaining an evaluation result of whether the contract to be evaluated lacks terms according to the classification result.
Furthermore, a trained softmax classifier is used for classifying the target sentence embedding expression matrix corresponding to the clause of the contract to be reviewed, so that the review result of whether the contract to be reviewed lacks the clause is obtained according to the classification result.
In some embodiments, the classifying the target sentence embedding expression matrix by using a pre-trained classification model, and obtaining an evaluation result of whether the contract to be evaluated lacks terms according to the classification result specifically includes: inputting the target sentence embedding vector into a pre-trained softmax classifier for classification prediction to obtain a probability value that the terms of the contract to be evaluated belong to a preset standard term category; and determining whether the clauses of the contract to be evaluated are missing according to the probability value.
The target sentence corresponding to the clause of the contract to be reviewed is embedded into the expression matrix and input into the trained softmax classifier for classification and prediction, and the prediction classification probability value output by the trained softmax classifier is obtained, wherein the prediction classification probability value represents the probability value of the clause of the contract to be reviewed belonging to the preset standard clause category, the closer the probability value is to 1, the case that the contract to be reviewed has no missing clause, and the closer the prediction classification probability value output by the trained softmax classifier is to 0, the case that the contract to be reviewed has missing clause.
For example, the predicted classification probability value output by the trained softmax classifier may be compared with a preset threshold, when the predicted classification probability value output by the trained softmax classifier exceeds the preset threshold, the review result indicates that the contract to be reviewed does not have missing terms, and when the predicted classification probability value output by the trained softmax classifier does not exceed the preset threshold, the review result indicates that the contract to be reviewed has missing terms, where the preset threshold may be 0.5.
And a trained softmax classifier is adopted to embed a target sentence corresponding to the clause of the contract to be reviewed into a representation matrix for classification prediction, so that the error is smaller, and the classification is more accurate.
The contract review method based on artificial intelligence provided by the above embodiment obtains a sample contract, performs syntactic analysis on the clauses of the sample contract, and obtains a syntactic analysis binary tree corresponding to the clauses of the sample contract; training the improved Transformer model with the enhanced attention mechanism according to the clause of the sample contract and the syntactic analysis binary tree corresponding to the clause to obtain a trained improved Transformer model; acquiring a contract to be evaluated and examined, and predicting a target sentence embedding expression matrix corresponding to the clause of the contract to be evaluated and examined by using the trained improved Transformer model; and classifying the target sentence embedding expression matrix by utilizing a pre-trained classification model, and obtaining an evaluation result of whether the contract to be evaluated lacks terms according to the classification result. Because the improved Transformer model enhances the original self-attention mechanism of the Transformer model, the attention distribution of the improved Transformer model is not dispersed, the training speed and the model precision can be effectively improved during training, the trained improved Transformer model can more quickly obtain a sentence embedding expression matrix capable of accurately representing the clause semantics of the contract to be reviewed, and the contract review is realized through the classification model on the basis of the sentence embedding expression matrix of the clause of the contract to be reviewed, so that the review efficiency and the review accuracy are greatly improved.
Referring to fig. 3, fig. 3 is a schematic block diagram of a contract review device based on artificial intelligence according to an embodiment of the present application.
As shown in fig. 3, the apparatus 300 includes: an analysis module 301, a training module 302, a prediction module 303, and a classification module 304.
The analysis module 301 is configured to obtain a sample contract, perform syntactic analysis on terms of the sample contract, and obtain a syntactic analysis binary tree corresponding to the terms of the sample contract;
the training module 302 is configured to train the improved Transformer model with the enhanced attention mechanism according to the clauses of the sample contract and the syntactic analysis binary tree corresponding to the clauses to obtain a trained improved Transformer model;
the prediction module 303 is configured to obtain a contract to be evaluated and to predict, by using the trained improved Transformer model, a target sentence embedding expression matrix corresponding to the clause of the contract to be evaluated;
the classification module 304 classifies the target sentence embedding expression matrix by using a pre-trained classification model, and obtains an evaluation result of whether the contract to be evaluated lacks terms according to the classification result.
It should be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the apparatus, the modules and the units described above may refer to the corresponding processes in the foregoing embodiment of the artificial intelligence based contract review method, and are not described herein again.
The apparatus provided by the above embodiments may be implemented in the form of a computer program, which can be run on a computer device as shown in fig. 4.
Referring to fig. 4, fig. 4 is a schematic block diagram of a computer device according to an embodiment of the present disclosure. The computer device may be a Personal Computer (PC), a server, or the like having a data processing function.
As shown in fig. 4, the computer device includes a processor, a memory, and a network interface connected by a system bus, wherein the memory may include a nonvolatile storage medium and an internal memory.
The non-volatile storage medium may store an operating system and a computer program. The computer program includes program instructions that, when executed, cause a processor to perform any one of the artificial intelligence based contract review methods.
The processor is used for providing calculation and control capability and supporting the operation of the whole computer equipment.
The internal memory provides an environment for the execution of a computer program on a non-volatile storage medium, which when executed by the processor, causes the processor to perform any one of the artificial intelligence based methods of contract review.
The network interface is used for network communication, such as sending assigned tasks and the like. Those skilled in the art will appreciate that the architecture shown in fig. 4 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
It should be understood that the Processor may be a Central Processing Unit (CPU), and the Processor may be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, etc. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Wherein, in one embodiment, the processor is configured to execute a computer program stored in the memory to implement the steps of:
obtaining a sample contract, and carrying out syntactic analysis on clauses of the sample contract to obtain a syntactic analysis binary tree corresponding to the clauses of the sample contract; training the improved Transformer model with the enhanced attention mechanism according to the clause of the sample contract and the syntactic analysis binary tree corresponding to the clause to obtain a trained improved Transformer model; acquiring a contract to be evaluated and examined, and predicting a target sentence embedding expression matrix corresponding to the clause of the contract to be evaluated and examined by using the trained improved Transformer model; and classifying the target sentence embedding expression matrix by utilizing a pre-trained classification model, and obtaining an evaluation result of whether the contract to be evaluated lacks terms according to the classification result.
In some embodiments, the processor is configured to implement the training of the attention mechanism-enhanced improved Transformer model according to the terms of the sample contract and the syntactic analysis binary tree corresponding to the terms of the sample contract, and when obtaining the trained improved Transformer model, the processor is configured to implement:
inputting the clauses of the sample contract and the syntactic analysis binary tree corresponding to the clauses into an improved Transformer model, so that the attention mechanism enhanced by the improved Transformer model generates a query matrix, a key matrix and a value matrix according to the clauses of the sample contract, and constructs a syntactic analysis mask matrix according to the syntactic analysis binary tree;
and training the improved Transformer model according to the query matrix, the key matrix, the value matrix and the syntactic analysis masking matrix to obtain the trained improved Transformer model.
In some embodiments, the processor is configured to, when the training of the improved Transformer model according to the query matrix, the key matrix, the value matrix, and the syntactic analysis masking matrix is performed to obtain a trained improved Transformer model, perform:
multiplying the query matrix and the key matrix to obtain a first attention matrix;
adding the attention matrix and the syntactic analysis covering matrix to obtain a second attention matrix;
blending the first attention matrix and the second attention matrix to obtain an enhanced new attention matrix;
multiplying the enhanced new attention matrix and the value matrix to obtain a sentence embedding expression matrix corresponding to the clause of the sample contract;
and updating the parameters of the improved Transformer model according to the sentence embedding expression matrix until the improved Transformer model converges to obtain the trained improved Transformer model.
In some embodiments, the processor is configured to, when performing the reconciliation on the first attention matrix and the second attention matrix to obtain the enhanced new attention matrix, perform:
acquiring a preset blending formula alpha A + (1-alpha) G, wherein alpha represents a preset door mechanism, A represents the first attention matrix, and G represents the second new attention matrix;
and substituting the first attention matrix and the second attention matrix into the preset harmonic formula for calculation to obtain an enhanced new attention matrix.
In some embodiments, the processor, when implementing the predicting, by using the trained improved Transformer model, the target sentence corresponding to the clause of the contract to be reviewed embedded into the representation matrix, is configured to implement:
performing syntactic analysis on the terms of the contract to be evaluated to obtain a target syntactic analysis binary tree corresponding to the terms of the contract to be evaluated;
and inputting the clauses of the contract to be evaluated and examined and the target syntactic analysis binary tree corresponding to the clauses into the trained improved Transformer model for prediction to obtain the target sentence embedding expression matrix.
In some embodiments, the processor, when implementing the syntactic analysis of the terms of the contract to be reviewed to obtain the target syntactic analysis binary tree corresponding to the terms of the contract to be reviewed, is configured to implement:
carrying out paragraph splitting on the contract to be evaluated to obtain the clause of the contract to be evaluated;
and carrying out syntactic analysis on the clauses of the contract to be reviewed by using a Stanford syntactic analysis tool to obtain the target syntactic analysis binary tree.
In some embodiments, the processor implements classification of the target sentence embedding expression matrix by using a pre-trained classification model, and when obtaining an evaluation result of whether the contract to be evaluated lacks terms according to the classification result, is configured to implement:
inputting the target sentence embedding vector into a pre-trained softmax classifier for classification prediction to obtain a probability value that the terms of the contract to be evaluated belong to a preset standard term category;
and determining whether the clauses of the contract to be evaluated are missing according to the probability value.
Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, where the computer program includes program instructions, and a method implemented when the program instructions are executed may refer to various embodiments of the artificial intelligence based contract review method according to the present application.
The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, for example, a hard disk or a memory of the computer device. The computer readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the computer device.
Further, the computer-readable storage medium may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created according to the use of the blockchain node, and the like.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments. While the invention has been described with reference to specific embodiments, the scope of the invention is not limited thereto, and those skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the invention. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. An artificial intelligence based contract review method, comprising the steps of:
obtaining a sample contract, and carrying out syntactic analysis on clauses of the sample contract to obtain a syntactic analysis binary tree corresponding to the clauses of the sample contract;
training the improved Transformer model with the enhanced attention mechanism according to the clause of the sample contract and the syntactic analysis binary tree corresponding to the clause to obtain a trained improved Transformer model;
acquiring a contract to be evaluated and examined, and predicting a target sentence embedding expression matrix corresponding to the clause of the contract to be evaluated and examined by using the trained improved Transformer model;
and classifying the target sentence embedding expression matrix by utilizing a pre-trained classification model, and obtaining an evaluation result of whether the contract to be evaluated lacks terms according to the classification result.
2. The artificial intelligence based contract review method of claim 1, wherein the training of the attention mechanism enhanced improved Transformer model according to the terms of the sample contract and the corresponding syntactic analysis binary tree thereof to obtain the trained improved Transformer model comprises:
inputting the clauses of the sample contract and the syntactic analysis binary tree corresponding to the clauses into an improved Transformer model, so that the attention mechanism enhanced by the improved Transformer model generates a query matrix, a key matrix and a value matrix according to the clauses of the sample contract, and constructs a syntactic analysis mask matrix according to the syntactic analysis binary tree;
and training the improved Transformer model according to the query matrix, the key matrix, the value matrix and the syntactic analysis masking matrix to obtain the trained improved Transformer model.
3. The artificial intelligence based contract review method of claim 2, wherein the training of the improved Transformer model according to the query matrix, the key matrix, the value matrix and the syntactic analysis masking matrix to obtain the trained improved Transformer model comprises:
multiplying the query matrix and the key matrix to obtain a first attention matrix;
adding the attention matrix and the syntactic analysis covering matrix to obtain a second attention matrix;
blending the first attention matrix and the second attention matrix to obtain an enhanced new attention matrix;
multiplying the enhanced new attention matrix and the value matrix to obtain a sentence embedding expression matrix corresponding to the clause of the sample contract;
and updating the parameters of the improved Transformer model according to the sentence embedding expression matrix until the improved Transformer model converges to obtain the trained improved Transformer model.
4. The artificial intelligence based contract review method of claim 3, wherein the reconciling the first attention matrix and the second attention matrix to obtain the enhanced new attention matrix comprises:
acquiring a preset blending formula alpha A + (1-alpha) G, wherein alpha represents a preset door mechanism, A represents the first attention matrix, and G represents the second new attention matrix;
and substituting the first attention matrix and the second attention matrix into the preset harmonic formula for calculation to obtain an enhanced new attention matrix.
5. The artificial intelligence based contract review method of claim 1, wherein the predicting, by using the trained improved Transformer model, the target sentence embedding representation matrix corresponding to the terms of the contract to be reviewed comprises:
performing syntactic analysis on the terms of the contract to be evaluated to obtain a target syntactic analysis binary tree corresponding to the terms of the contract to be evaluated;
and inputting the clauses of the contract to be evaluated and examined and the target syntactic analysis binary tree corresponding to the clauses into the trained improved Transformer model for prediction to obtain the target sentence embedding expression matrix.
6. The artificial intelligence based contract review method of claim 5, wherein the syntactic analyzing the clauses of the contract to be reviewed to obtain a target syntactic analysis binary tree corresponding to the clauses of the contract to be reviewed comprises:
carrying out paragraph splitting on the contract to be evaluated to obtain the clause of the contract to be evaluated;
and carrying out syntactic analysis on the clauses of the contract to be reviewed by using a Stanford syntactic analysis tool to obtain the target syntactic analysis binary tree.
7. The artificial intelligence based contract review method according to claim 1, wherein the classifying the target sentence embedding expression matrix by using a pre-trained classification model, and obtaining a review result of whether the contract to be reviewed lacks terms according to the classification result comprises:
inputting the target sentence embedding vector into a pre-trained softmax classifier for classification prediction to obtain a probability value that the terms of the contract to be evaluated belong to a preset standard term category;
and determining whether the clauses of the contract to be evaluated are missing according to the probability value.
8. An artificial intelligence based contract review apparatus, comprising:
the analysis module is used for obtaining a sample contract, and performing syntactic analysis on clauses of the sample contract to obtain a syntactic analysis binary tree corresponding to the clauses of the sample contract;
the training module is used for training the improved Transformer model with the enhanced attention mechanism according to the clauses of the sample contract and the syntactic analysis binary tree corresponding to the clauses to obtain a trained improved Transformer model;
the prediction module is used for acquiring a contract to be evaluated and examined and predicting a target sentence embedding expression matrix corresponding to the clause of the contract to be evaluated and examined by using the trained improved Transformer model;
and the classification module is used for classifying the target sentence embedding expression matrix by using a pre-trained classification model and obtaining an evaluation result of whether the contract to be evaluated lacks terms according to the classification result.
9. A computer device comprising a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein the computer program, when executed by the processor, performs the steps of the artificial intelligence based contract review method of any of claims 1-7.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, wherein the computer program, when being executed by a processor, carries out the steps of the artificial intelligence based contract review method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111152131.8A CN113887195A (en) | 2021-09-29 | 2021-09-29 | Contract review method, device, equipment and storage medium based on artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111152131.8A CN113887195A (en) | 2021-09-29 | 2021-09-29 | Contract review method, device, equipment and storage medium based on artificial intelligence |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113887195A true CN113887195A (en) | 2022-01-04 |
Family
ID=79008156
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111152131.8A Pending CN113887195A (en) | 2021-09-29 | 2021-09-29 | Contract review method, device, equipment and storage medium based on artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113887195A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115271553A (en) * | 2022-09-27 | 2022-11-01 | 湖南华菱电子商务有限公司 | Contract management method and device based on big data, electronic equipment and storage medium |
-
2021
- 2021-09-29 CN CN202111152131.8A patent/CN113887195A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115271553A (en) * | 2022-09-27 | 2022-11-01 | 湖南华菱电子商务有限公司 | Contract management method and device based on big data, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Spedicato | Discrete Time Markov Chains with R. | |
| WO2021042503A1 (en) | Information classification extraction method, apparatus, computer device and storage medium | |
| CN109885824B (en) | Hierarchical Chinese named entity recognition method, hierarchical Chinese named entity recognition device and readable storage medium | |
| US11521075B2 (en) | Transfer learning system for automated software engineering tasks | |
| CN112347310A (en) | Event processing information query method and device, computer equipment and storage medium | |
| WO2022048363A1 (en) | Website classification method and apparatus, computer device, and storage medium | |
| CN112131888B (en) | Method, device, equipment and storage medium for analyzing semantic emotion | |
| CN110134780B (en) | Method, device, equipment and computer readable storage medium for generating document abstract | |
| CN112100401A (en) | Knowledge graph construction method, device, equipment and storage medium for scientific and technological service | |
| CN112686049A (en) | Text auditing method, device, equipment and storage medium | |
| Sankoff | Probability and linguistic variation | |
| CN112860919A (en) | Data labeling method, device and equipment based on generative model and storage medium | |
| CN113887195A (en) | Contract review method, device, equipment and storage medium based on artificial intelligence | |
| Yan et al. | Cross-site scripting attack detection based on a modified convolution neural network | |
| CN111767720B (en) | Title generation method, computer and readable storage medium | |
| CN112765966A (en) | Method and device for removing duplicate of associated word, computer readable storage medium and electronic equipment | |
| CN111507108B (en) | Alias generation method and device, electronic equipment and computer readable storage medium | |
| CN112446205A (en) | Sentence distinguishing method, device, equipment and storage medium | |
| Lytvyn et al. | Content Formation Method in the Web Systems. | |
| CN114900346B (en) | Network security testing method and system based on knowledge graph | |
| CN112948582B (en) | Data processing method, device, equipment and readable medium | |
| CN111562943B (en) | Code clone detection method and device based on event embedded tree and GAT network | |
| CN114925197A (en) | Deep learning text classification model training method based on topic attention | |
| CN117807603B (en) | Software supply chain auditing method, system and computer readable storage medium | |
| WO2020058120A1 (en) | Method and apparatus for determining existence of dependence violation, electronic device, medium and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |