CN112613294A - Method and device for inspecting judgment result of legal document - Google Patents
Method and device for inspecting judgment result of legal document Download PDFInfo
- Publication number
- CN112613294A CN112613294A CN201910887589.4A CN201910887589A CN112613294A CN 112613294 A CN112613294 A CN 112613294A CN 201910887589 A CN201910887589 A CN 201910887589A CN 112613294 A CN112613294 A CN 112613294A
- Authority
- CN
- China
- Prior art keywords
- document
- criminal
- target
- referee
- result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000013528 artificial neural network Methods 0.000 claims abstract description 82
- 238000013145 classification model Methods 0.000 claims abstract description 82
- 238000012795 verification Methods 0.000 claims abstract description 16
- 239000013598 vector Substances 0.000 claims description 91
- 238000012549 training Methods 0.000 claims description 60
- 230000011218 segmentation Effects 0.000 claims description 18
- 230000006870 function Effects 0.000 claims description 9
- 230000003044 adaptive effect Effects 0.000 claims description 6
- 238000012360 testing method Methods 0.000 claims description 5
- 238000004891 communication Methods 0.000 claims description 3
- 238000010998 test method Methods 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 9
- 238000004590 computer program Methods 0.000 description 6
- 238000002372 labelling Methods 0.000 description 6
- 230000008901 benefit Effects 0.000 description 4
- 238000012790 confirmation Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 210000000056 organ Anatomy 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013524 data verification Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The invention discloses a method and a device for inspecting a judge result of a legal document, wherein the method comprises the steps of determining a target criminal judge document and extracting target document contents from the document, wherein the contents of a template document can influence the contents of the judge result; inputting the target document content into a pre-constructed neural network classification model to obtain a judgment result processed by the neural network classification model; and comparing the referee result processed by the neural network classification model with the referee result extracted and processed from the target criminal referee document to obtain a verification result. In the method, the neural network classification model is a machine model trained by using a large number of document samples, and the output judgment result depends on the judgment experience of the large number of document samples and can be used as a standard for checking the artificial judgment result in the target criminal judgment document.
Description
Technical Field
The invention relates to the technical field of data verification, in particular to a method and a device for testing a judgment result of a legal document.
Background
When a court officer deals with a criminal case, the court officer generates a criminal official document according to a litigation request of a public institution, the fact confirmation made by the criminal officer on the criminal case and the final judgment result of the criminal officer on the criminal case.
Currently, the referee results in criminal referee documents are generated by court judges according to the specifics of the criminal case. While the judge judges the criminal case, the judgment of the criminal case may be deviated due to excessive factors and other reasons.
Therefore, a method for checking the result of judge of legal documents is needed to check whether the judge result of court officer on criminal case has deviation.
Disclosure of Invention
In view of the above, the present invention provides a method of verifying the results of a judge of a legal document that overcomes or at least partially solves the above problems. In addition, the invention also provides a device for inspecting the result of the judge of the legal document, so as to ensure the application and the realization of the method in practice.
In a first aspect, the invention provides a method for inspecting the result of a judge of a legal document, which comprises the following steps:
determining a target criminal referee document;
acquiring target document contents from the target criminal referee documents, wherein the target document contents are contents capable of influencing referee results;
inputting the target document content into a pre-constructed neural network classification model to obtain a judgment result which is output by the neural network classification model and is adaptive to the target document content;
and comparing the judgment result output by the neural network classification model with the judgment result in the target criminal judgment document to obtain a verification result.
In a second aspect, the present invention provides a device for checking the result of a judge of a legal document, comprising:
a document determination unit for determining a target criminal referee document;
a content acquisition unit for acquiring target document contents from the target criminal referee document, wherein the target document contents are contents capable of influencing the referee result;
the judging result acquisition unit is used for inputting the target document content into a pre-constructed neural network classification model so as to obtain a judging result which is output by the neural network classification model and is adaptive to the target document content;
and the judging result checking unit is used for comparing the judging result output by the neural network classification model with the judging result in the target criminal judging document to obtain a checking result.
In a third aspect, the present invention provides a storage medium having stored thereon a program which, when executed by a processor, implements the above-described method for checking the result of a official document of a legal instrument.
In a fourth aspect, the present invention provides an electronic device comprising at least one processor, and at least one memory connected to the processor, a bus; the processor and the memory complete mutual communication through the bus; the processor is used for calling the program instructions in the memory so as to execute the method for checking the result of the legal document referee.
By means of the technical scheme, the method for inspecting the legal document referee result provided by the invention comprises the steps of determining a target criminal referee document and extracting target document contents from the document, wherein the template document contents can influence the contents of the referee result; inputting the target document content into a pre-constructed neural network classification model to obtain a judgment result processed by the neural network classification model; and comparing the referee result processed by the neural network classification model with the referee result extracted and processed from the target criminal referee document to obtain a verification result. In the method, the neural network classification model is a machine model trained by using a large number of document samples, and the output judgment result depends on the judgment experience of the large number of document samples and can be used as a standard for checking the artificial judgment result in the target criminal judgment document.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a flow chart showing a method for checking the result of a judge of a legal document according to the present invention;
FIG. 2 is a flow chart illustrating the construction of a neural network classification model provided by the present invention;
FIG. 3 is a block diagram showing the structure of a legal document referee result verifying device provided by the invention;
FIG. 4 is a block diagram illustrating a neural network classification model building structure provided by the present invention;
fig. 5 shows a block diagram of the device structure provided by the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
When a court officer deals with a criminal case, the court officer can generate a criminal official document according to a litigation request of a public complaint organ, the fact confirmation made by the court officer on the criminal case and the final judgment result of the criminal officer on the criminal case, wherein the criminal official document represents the criminal state committed by an officer, the criminal responsibility assumed by the criminal state and the criminal penalty official document corresponding to the criminal responsibility.
Currently, the referee results in criminal referee documents are generated by court judges according to the specifics of the criminal case. While the judge judges the criminal case, the judgment of the criminal case may be deviated due to excessive factors and other reasons. For example: the method has the advantages that the method has deviation in conviction of the defendant, wrong use of penalty for the defendant, non-standard criminal period scale of the defendant and the like.
Therefore, the invention provides a method for inspecting the result of a legal official document, which is used for verifying the target criminal official document. Referring to fig. 1, the method specifically includes S101-S104. Wherein:
s101: and determining a target criminal referee document.
Specifically, the target criminal referee document is a referee document to be verified. Before the target criminal referee document is verified, any criminal referee document can be determined as the target criminal referee document according to actual requirements.
It should be noted that the target criminal official document may include, but is not limited to, the following items of information: the content of the defendant information part, the content of the litigation request part, the content of the fact affirmation part, the content of the certificate part, and the content of the referee result part. Wherein the content of the referee results part of the target criminal referee document is used to participate in the verification step, as detailed below.
S102: and acquiring target document contents from the target criminal referee documents, wherein the target document contents are contents capable of influencing the referee results.
It should be noted that the target document content is a part of the target criminal official document, and this part of the content is a content that can affect the official result. For example, the target document content may specifically include: any one or more of the contents of the litigation-request section, the contents of the fact-identified section, and the contents of the referee-result section. The litigation request is made by the party to the court, and the court is required to make corresponding judgment on the request.
Specifically, the target document content is obtained from the target criminal referee document. For example: analyzing the content of the target document from the target criminal referee document by using a natural language processing technology, such as analyzing the content of a litigation request part; for another example, the target criminal referee document has a specific format specification, each part of content in the document has a content identifier, and the document content corresponding to the internal identifier is extracted according to the content identifier, so that the target document content can be obtained.
In one example, acquiring any one or more of the content of the litigation request section, the content of the fact affirmation section, and the content of the referee result section as the target document content from the target criminal referee document includes:
searching the content of the litigation request part in the target criminal referee document; if the content of the litigation request part is found, determining the content of the litigation request part as the content of the target document; if the content of the litigation request part is not found, finding the content of the fact affirmation part in the target criminal referee document; if the content of the fact affirming part is found, the content of the fact affirming part is determined as the target document content; and if the content of the factual affirmation part is not found, determining the content of the judge result part in the target criminal judge document as the content of the target document.
Specifically, the content of the litigation request part, the content of the fact identification part and the content of the judgment result part are acquired from the target criminal referee document, and the priority of the contents exists in the sequence, namely the highest priority is the content of the litigation request part, the second is the content of the fact identification part and the last is the content of the judgment result part. The reason why the content of the litigation request part is the highest priority is that the part is the original data submitted by the public prosecution agency and the court does not participate in the original data, so that when the part is used as the target document content, the deviation is the smallest and the accuracy is the highest, and the part is the content of the fact affirmation part and the content of the judge result part.
Step S103 is executed according to the acquired target document content.
S103: and inputting the target document content into a pre-constructed neural network classification model to obtain a judgment result which is output by the neural network classification model and is adaptive to the target document content.
It should be noted that the pre-constructed neural network classification model may be an RCNN (Region-CNN) algorithm model or a BERT algorithm model, etc. The neural network classification model is used for carrying out model operation on input target document contents to obtain a judgment result corresponding to the target document contents. That is, the neural network classification model is a machine model obtained by training a large number of criminal official document samples, and is specifically used for determining the official result adapted to the content of the target document.
Specifically, after the target document content is input into the neural network classification model, the neural network classification model performs word segmentation on the target document content to obtain a plurality of word segmentation results corresponding to the target document content, then generates corresponding word vectors for each word segmentation result, sequentially splices the word vectors of the plurality of words to generate spliced vectors, and then determines a judgment result corresponding to the vector from the neural network classification model to serve as a prediction judgment result of the target document content.
S104: and comparing the judgment result output by the neural network classification model with the judgment result in the target criminal judgment document to obtain a verification result.
It should be noted that the referee result includes any one or more of the following items: the name of the conviction, the suitable criminal species and the cutting amount of the criminal period.
The referee results in the target criminal referee document can be extracted from the document using regular expressions. Specifically, the referee result output by the neural network classification model is compared with the referee result in the target criminal referee document:
and if the judging result in the target criminal judging document is consistent with the judging result output by the neural network classification model, the target criminal judging document is free of deviation.
And if the judging result in the target criminal judging document is inconsistent with the judging result output by the neural network classification model, indicating that the target criminal judging document has deviation. For example: the referee results in the target criminal referee document are: stealing crime and apprenticating 10 months, and the judgment result obtained by the neural network classification model is as follows: robbing a crime, and carrying out apprentication for 10 months, wherein the crime name deviation of the judgment result in the target criminal judge document is shown in the case; if the judgment result obtained by the neural network classification model is a crime of stealing + robbing, and the criminal has a period of 10 months, the judgment result is compared with the judgment result of the analysis target criminal judgment document, and the situation also belongs to the crime name deviation.
In addition, the inconsistency between the referee result analyzed from the target criminal referee document and the referee result output by the neural network classification model is also shown in: criminal seed deviation and referee criminal phase deviation. For example: the judgment result in the target criminal referee document is social management order crime, the period of the arrest is 2 months, the judgment result obtained through the neural network classification model is social management order crime, the period of the control is 1 month, and the situation indicates that the judgment result in the target criminal referee document has criminal deviation and criminal period deviation.
The method comprises the steps that the judgment result obtained by analyzing a target criminal referee document is set according to the judgment result obtained by a neural network classification model, and if the judgment result obtained by analyzing the target criminal referee document and the judgment result obtained by the neural network classification model only have a criminal name deviation, criminal seeds and criminal periods are consistent, and the criminal period has a deviation; if there is only criminal seed deviation, the name of the crime and the criminal period are consistent, and there is a criminal seed deviation; if there is only criminal period deviation, the criminal name and the criminal species are consistent, and there is deviation in the criminal period. In addition, the target criminal referee documents can have a plurality of deviation items, such as the criminal names and criminal species have deviation, and the criminal periods are consistent; the name of the crime, the criminal period have deviation, the criminal species are consistent; the criminal period and the criminal species have deviation and the crime names are consistent; the name, the criminal stage and the criminal species of the conviction have deviations.
In order to examine a judge organ (court), it is necessary to count criminal referee documents of various criminal cases and count the deviation number of the criminal referee documents. The following methods are provided for this purpose:
in one example, in the method for checking the result of a law official document, the target criminal official document may be plural, and the method further comprises:
and if the verification result shows that the deviation exists, respectively counting the ratio of the target criminal referee documents with different deviation types to the total amount of the target criminal referee documents according to the deviation types.
Specifically, if there is a deviation between the referee result analyzed from the target criminal referee document and the referee result output by the neural network classification model, the ratio of the target criminal referee document of the deviation type with the deviation to the total amount of the target criminal referee document is counted. Wherein the types of deviation include: criminal name deviation, criminal species deviation or referee criminal period deviation. According to the method, the rate of occurrence of certain types of deviation in certain time can be counted.
In one example, where the target criminal referee document is in plurality and the target criminal referee document has a corresponding classification type, the method further comprises:
and respectively counting the ratio of the target criminal official documents corresponding to different verification results under different classification types to the total amount of the target criminal official documents according to the classification types.
It should be noted that the classification types of the target criminal referee documents include: and judging the court, the area where the court is located and other classification standards.
Specifically, if there is a deviation between the referee result obtained by analysis from the target criminal referee document and the referee result output by the neural network classification model, the ratio of the target criminal referee document corresponding to the classification type to the total amount of the target criminal referee document is counted. For example: and when the classification type is a judgment court, counting the ratio of the target criminal referee document with judgment deviation to the total amount of the target criminal referee document by different judgment courts, or further counting the ratio of the target criminal referee document with different deviation types to the total amount of the target criminal referee document by different judgment courts.
According to the technical scheme, the invention provides a method for testing the judgment result of a legal document, which comprises the steps of determining a target criminal judgment document and extracting the content of the target document from the document, wherein the content of a template document can influence the content of the judgment result; inputting the target document content into a pre-constructed neural network classification model to obtain a judgment result processed by the neural network classification model; and comparing the referee result processed by the neural network classification model with the referee result extracted and processed from the target criminal referee document to obtain a verification result. In the method, the neural network classification model is a machine model trained by using a large number of document samples, and the output judgment result depends on the judgment experience of the large number of document samples and can be used as a standard for checking the artificial judgment result in the target criminal judgment document.
In one example, a neural network classification model is constructed. Referring to fig. 2, the method specifically includes steps S201 to S203. Wherein:
s201: and obtaining a criminal referee document training sample which has a manually marked referee result.
It should be noted that, in order to ensure that the neural network classification model outputs a referee result with high accuracy, a large number of criminal referee document training samples corresponding to the manually labeled referee result need to be input into the neural network classification model for training.
Specifically, a large number of criminal referee documents are downloaded from a network (such as a referee document network) and used as criminal referee document training samples of a neural network classification model, the referee results of all the criminal referee document training samples are manually marked, and the criminal names, the applicable criminal species and the referee amount and the criminal period in the referee results are marked. The name of a conviction is taken as an example: assuming that 50 types of crime names appear in a certain batch of criminal referee document training samples, manually marking the samples as 50-dimensional crime name vectors, wherein each dimension in the crime name vectors expresses one crime name, and when the dimension is 0, the crime name is not expressed, and when the dimension is 1, the crime name is expressed. For example: the criminal name vector corresponding to the conviction name of a criminal referee paper training sample is {0, 0, 0, … …, 1}, wherein only the last 1 bit in the vector is 1, which indicates that the criminal referee paper training sample only has a criminal name with the last dimension. The vector can be called a one-dimensional labeling vector of the judgment result.
Besides the criminal name vector, the applicable criminal species vector and the referee criminal period vector are generated according to the marking method, and the vector generated by the marking method can also be called a referee result one-dimensional marking vector.
The mapping relationship between the dimension and the guilty name can be freely set by the administrator, and is not specifically described here.
S202: inputting the criminal referee document training sample into the initial neural network classification model so as to enable the initial neural network classification model to execute the step of predicting the referee result.
It should be noted that the initial neural network classification model is an untrained algorithm model, and parameter values in the model are initial values. Specifically, the step of predicting the referee's result may include the following steps a1 to a 4.
A1: and performing word segmentation operation on the criminal referee document training sample to obtain a plurality of words.
Specifically, after the criminal referee document training sample is subjected to word segmentation by the word segmentation tool, a plurality of word segments corresponding to the criminal referee document are obtained. The word segmentation tool for segmenting words of criminal referee documents can be any one, and is not specifically described herein.
A2: and respectively carrying out word vector training on each word segmentation to obtain a word vector of each word segmentation.
It should be noted that the process of performing word vector training on a word is performed in a word vector training model (e.g., word2vector), where the word vector training model includes a trained corpus, and each corpus in the corpus is mapped with a vector of a preset dimension.
Specifically, each participle is input into a word vector training model, a corpus corresponding to each participle is determined, and then a word vector corresponding to each participle is determined. For example: after word segmentation, the word capture crime is obtained, the word segmentation result is used for finding out the corresponding corpus and the word vector corresponding to the corpus according to the word segmentation result, and the word vector is used as the word vector of the word segmentation of the capture crime.
It should be noted that the word vector training model has a preset unknown word vector, and the word vector is used to represent a word segmentation that cannot be recognized by the corpus. If the preset unknown word vector is: and UNK (unknown keyword) for representing all the word segments which can not be identified by the corpus by using the word vector, wherein the dimension of the word vector is consistent with the dimension of the word vector corresponding to the corpus.
A3: and performing connection operation on all word vectors of the criminal referee training sample to obtain a referee result one-dimensional training vector.
Specifically, after the criminal referee training sample is participled in the step A1, a plurality of participles are obtained; the word vector of the preset dimension corresponding to each participle is determined through the step A2. If n participles are obtained after the word segmentation step, determining a word vector with m dimensions corresponding to each participle, and connecting all the word vectors to obtain a vector with n x m dimensions. The n × m-dimensional vector needs to be subjected to a concatenation operation to obtain a one-dimensional vector, which may be referred to as a one-dimensional training vector of the referee result for convenience of description.
It should be noted that the connection model includes: CNN layer, RNN (or LSTM) layer, and full connection layer, etc. After the connection model performs the connection operation, the dimension of the output one-dimensional training vector of the referee result needs to be the same as the dimension of the vector generated by the above-mentioned label (i.e. the one-dimensional label vector of the referee result). Therefore, parameters of the connection model can be set according to the one-dimensional labeling vector of the judgment result, specifically, parameters in the full connection layer are set, so that the purpose that the output dimensions are the same is achieved.
It should be noted that the one-dimensional label vector of the referee result and the content of the referee result have a corresponding relationship, that is, the referee result of each content has its own one-dimensional label vector of the referee result. The content of the referee result comprises: the name of the conviction, the applicable criminal species and the referee criminal period. Therefore, the word vectors of the referee result of each content need to be connected to obtain the one-dimensional training vector of the referee result of each content.
Firstly, the generation mode of the conviction name vector is as follows: and setting parameters of a connection model according to the dimensionality of a marking vector of the crime name, and inputting an n1 × m1 dimensional criminal referee training sample document vector into the connection model to obtain a one-dimensional training vector of the crime name.
Then, the generation method of the applicable criminal seed vector is as follows: and setting parameters of the connection model according to the dimensionality of the labeling vector of the applicable criminal seeds, and inputting the n2 × m2 dimensional criminal referee training sample document vector into the connection model to obtain the one-dimensional training vector of the applicable criminal seeds.
Finally, the generation mode of the cutting penalty period vector is as follows: and setting parameters of a connection model according to the dimensionality of the labeling vector of the measuring criminal period, and inputting the n3 × m3 dimensional criminal referee training sample document vector into the connection model to obtain the one-dimensional training vector of the measuring criminal period.
It should be noted that the dimension of the one-dimensional training vector of the referee result is consistent with the dimension of the one-dimensional labeling vector of the referee result. The sequence of the one-dimensional training vectors for generating the referee results with different contents is not limited to this, and may be any other sequence, or may be any two or any three of them executed in parallel.
A4: and determining a target element value reaching a preset threshold value in the one-dimensional training vector of the judgment result, and obtaining a predicted judgment result corresponding to the target element value.
Specifically, the one-dimensional training vector of the judgment result is converted into a probability vector, that is, the one-dimensional training vector of the judgment result is input into a probability vector conversion function (such as a sigmoid function) to obtain a probability vector corresponding to the one-dimensional training vector of the judgment result, each element in the probability vector corresponds to a target element value, and when the target element value of a certain element reaches a preset threshold value, the judgment result is used for the element.
For example: after the name vector of the conviction is converted into the probability vector, an element (robbery) with a target element value of 0.6 and an element (theft conviction) with a target element value of 0.4 are provided, the target element value in the vector is compared with a preset threshold value, the preset threshold value is assumed to be 0.5, the robbery can be determined to be output after the target element value is compared with the preset threshold value, and the theft conviction is not output. In this way the predicted referee results for the criminal referee's paper training samples are determined.
S203: and comparing the predicted judgment result with the manually marked judgment result, and adjusting the parameter value of the initial neural network classification model according to the comparison result until the output predicted judgment result meets the requirement of a preset loss function, so as to obtain the trained neural network classification model.
Specifically, the predicted judgment result is compared with the manually marked judgment result, the parameter value of the initial neural network classification model is adjusted according to the comparison result, the adjusted neural network classification model is trained continuously until the error between the output predicted judgment result and the manually marked judgment result can meet the requirement of the loss function, and therefore training of the neural network classification model is completed.
The invention provides a device for inspecting the judgment result of a legal document. Referring to fig. 3, the apparatus includes: document determining section 301, content acquiring section 302, referee result acquiring section 303, and referee result verifying section 304. Wherein:
a document determination unit 301 for determining a target criminal referee document.
A content obtaining unit 302, configured to obtain target document content from a target criminal referee document, wherein the target document content is content capable of affecting the referee result.
The referee result obtaining unit 303 is configured to input the target document content into a pre-constructed neural network classification model, so as to obtain a referee result that is output by the neural network classification model and is adapted to the target document content.
And the referee result checking unit 304 is configured to compare the referee result output by the neural network classification model with the referee result in the target criminal referee document to obtain a checking result.
According to the technical scheme, the device for testing the result of the legal document referee determines the target criminal referee document and extracts the target document content from the document, wherein the content of the template document can influence the content of the referee result; inputting the target document content into a pre-constructed neural network classification model to obtain a judgment result processed by the neural network classification model; and comparing the referee result processed by the neural network classification model with the referee result extracted and processed from the target criminal referee document to obtain a verification result. In the device, the neural network classification model is a machine model trained by using a large number of document samples, and the output judgment result depends on the judgment experience of the large number of document samples and can be used as a standard for checking the artificial judgment result in the target criminal judgment document.
In one example, when the content acquiring unit acquires the target document content from the target criminal referee document, the content acquiring unit is specifically configured to:
any one or more of the contents of the litigation request section, the contents of the fact confirmation section, and the contents of the referee result section are acquired from the target criminal referee document as the target document contents.
In one example, the content acquiring unit acquires, as the target document content, any one or more of the content of the litigation requesting part, the content of the fact recognizing part, and the content of the referee result part from the target criminal referee document, and is specifically configured to:
searching the content of the litigation request part in the target criminal referee document; if the content of the litigation request part is found, determining the content of the litigation request part as the content of the target document; if the content of the litigation request part is not found, finding the content of the fact affirmation part in the target criminal referee document; if the content of the fact affirming part is found, the content of the fact affirming part is determined as the target document content; and if the content of the factual affirmation part is not found, determining the content of the judge result part in the target criminal judge document as the content of the target document.
In one example, referring to fig. 4, the apparatus for verifying the result of the official document of law further comprises:
and the referee result labeling unit 401 is used for obtaining a criminal referee document training sample which has a manually labeled referee result.
A referee result prediction unit 402, configured to input criminal referee document training samples into the initial neural network classification model, so that the initial neural network classification model performs the following steps of predicting referee results:
performing word segmentation operation on the criminal referee document training sample to obtain a plurality of words; respectively carrying out word vector training on each participle to obtain a word vector of each participle; performing connection operation on all word vectors of the criminal referee training sample to obtain a referee result one-dimensional training vector; determining a target element value reaching a preset threshold value in the one-dimensional training vector of the judgment result, and obtaining a predicted judgment result corresponding to the target element value;
a judge result comparing unit 403, configured to compare the predicted judge result with a manually labeled judge result, and adjust a parameter value of the initial neural network classification model according to the comparison result until the output predicted judge result meets a requirement of a preset loss function, so as to obtain a trained neural network classification model.
In one example, the referee results include any one or more of: the name of the conviction, the suitable criminal species and the cutting amount of the criminal period.
In one example, the target criminal official document is plural, and the checking device for the legal document official result further comprises:
and the first ratio calculating unit is used for respectively counting the ratio of the target criminal referee documents with different deviation types to the total amount of the target criminal referee documents according to the deviation types if the verification result shows that the deviation exists.
In one example, the target criminal referee document is plural and has a corresponding classification type, the apparatus for inspecting the result of legal document referee further comprises:
and the second ratio calculating unit is used for respectively counting the ratio of the target criminal official document corresponding to different checking results under different classification types to the total amount of the target criminal official document according to the classification types.
The legal document referee result checking device comprises a processor and a memory, wherein the document determining unit, the content acquiring unit, the referee result checking unit and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more than one, and the judgment result is verified by adjusting the kernel parameters.
An embodiment of the present invention provides a storage medium having a program stored thereon, the program implementing the method for checking the result of a judge of a legal document when executed by a processor.
The embodiment of the invention provides a processor, which is used for running a program, wherein the program runs to execute the method for checking the result of the legal document referee.
The embodiment of the invention provides equipment, which comprises at least one processor, at least one memory and a bus, wherein the memory and the bus are connected with the processor; the processor and the memory complete mutual communication through a bus; the processor is used for calling the program instructions in the memory so as to execute the method for checking the result of the legal document referee. The device herein may be a server, a PC, a PAD, a mobile phone, etc. Referring to fig. 5, one configuration of the device 50 is shown, the device 50 including a processor 501, a memory 502, and a bus 503, wherein the processor 501 and the memory 502 are coupled via the bus 503.
The invention also provides a computer program product adapted to perform a program for initializing the following method steps when executed on a data processing device:
and determining a target criminal referee document.
And acquiring target document contents from the target criminal referee documents, wherein the target document contents are contents capable of influencing the referee results.
And inputting the target document content into a pre-constructed neural network classification model to obtain a judgment result which is output by the neural network classification model and is adaptive to the target document content.
And comparing the judgment result output by the neural network classification model with the judgment result in the target criminal judgment document to obtain a verification result.
In one example, obtaining target document content from a target criminal referee document comprises:
any one or more of the contents of the litigation request section, the contents of the fact confirmation section, and the contents of the referee result section are acquired from the target criminal referee document as the target document contents.
In one example, acquiring any one or more of the content of the litigation request section, the content of the fact affirmation section, and the content of the referee result section as the target document content from the target criminal referee document includes:
and searching the content of the litigation request part in the target criminal referee document.
And if the content of the litigation request part is found, determining the content of the litigation request part as the content of the target document.
If the content of the litigation request part is not found, the content of the fact affirmation part is found in the target criminal referee document.
If the contents of the fact-identified portion are found, the contents of the fact-identified portion are determined as the contents of the target document.
And if the content of the factual affirmation part is not found, determining the content of the judge result part in the target criminal judge document as the content of the target document.
In one example, the neural network classification model is constructed in a manner that includes: and obtaining a criminal referee document training sample which has a manually marked referee result.
Inputting the criminal referee document training sample into the initial neural network classification model so that the initial neural network classification model executes the following steps of predicting referee results:
performing word segmentation operation on the criminal referee document training sample to obtain a plurality of words; respectively carrying out word vector training on each participle to obtain a word vector of each participle; performing connection operation on all word vectors of the criminal referee training sample to obtain a referee result one-dimensional training vector; and determining a target element value reaching a preset threshold value in the one-dimensional training vector of the judgment result, and obtaining a predicted judgment result corresponding to the target element value.
And comparing the predicted judgment result with the manually marked judgment result, and adjusting the parameter value of the initial neural network classification model according to the comparison result until the output predicted judgment result meets the requirement of a preset loss function, so as to obtain the trained neural network classification model.
In one example, the referee results include any one or more of: the name of the conviction, the suitable criminal species and the cutting amount of the criminal period.
In one example, the target criminal referee document is plural, then the method further comprises:
and if the verification result shows that the deviation exists, respectively counting the ratio of the target criminal referee documents with different deviation types to the total amount of the target criminal referee documents according to the deviation types.
In one example, where the target criminal referee document is in plurality and the target criminal referee document has a corresponding classification type, the method further comprises:
and respectively counting the ratio of the target criminal official documents corresponding to different verification results under different classification types to the total amount of the target criminal official documents according to the classification types.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a device includes one or more processors (CPUs), memory, and a bus. The device may also include input/output interfaces, network interfaces, and the like.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip. The memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The above are merely examples of the present invention, and are not intended to limit the present invention. Various modifications and alterations to this invention will become apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.
Claims (10)
1. A method for testing the result of a judge of a legal document is characterized by comprising the following steps:
determining a target criminal referee document;
acquiring target document contents from the target criminal referee documents, wherein the target document contents are contents capable of influencing referee results;
inputting the target document content into a pre-constructed neural network classification model to obtain a judgment result which is output by the neural network classification model and is adaptive to the target document content;
and comparing the judgment result output by the neural network classification model with the judgment result in the target criminal judgment document to obtain a verification result.
2. The method for verifying the result of a judge of a legal document according to claim 1, wherein obtaining the contents of a target document from said target criminal referee document comprises:
and acquiring any one or more of the content of the litigation request part, the content of the fact affirmation part and the content of the referee result part from the target criminal referee document as the target document content.
3. The method for verifying the result of a judge of a legal document according to claim 2, wherein the step of acquiring any one or more of the contents of a litigation request section, the contents of a fact recognition section, and the contents of a judge result section from the target criminal referee document as the target document contents comprises:
searching the content of the litigation request part in the target criminal referee document;
if the content of the litigation request part is found, determining the content of the litigation request part as the content of the target document;
if the content of the litigation request part is not found, finding the content of the fact affirmation part in the target criminal referee document;
if the content of the fact affirming part is found, determining the content of the fact affirming part as the target document content;
and if the content of the fact-identified part is not found, determining the content of the judge result part in the target criminal judge document as the content of the target document.
4. The method for testing the result of a judge of a legal document according to claim 1, wherein the neural network classification model is constructed in a manner comprising:
obtaining a criminal referee document training sample, wherein the criminal referee document training sample has a manually marked referee result;
inputting the criminal referee document training sample into an initial neural network classification model so that the initial neural network classification model executes the following steps of predicting referee results:
performing word segmentation operation on the criminal referee document training sample to obtain a plurality of words;
respectively carrying out word vector training on each participle to obtain a word vector of each participle;
performing connection operation on all word vectors of the criminal referee training sample to obtain a referee result one-dimensional training vector;
determining a target element value reaching a preset threshold value in the one-dimensional training vector of the judgment result, and obtaining a predicted judgment result corresponding to the target element value; and comparing the predicted judgment result with a manually marked judgment result, and adjusting the parameter value of the initial neural network classification model according to the comparison result until the output predicted judgment result meets the requirement of a preset loss function, so as to obtain the trained neural network classification model.
5. The method of claim 1, wherein the official result of the legal document comprises any one or more of the following: the name of the conviction, the suitable criminal species and the cutting amount of the criminal period.
6. The method for verifying the result of a judge of a legal document according to claim 1, wherein said target criminal official document is plural, and further comprising:
and if the verification result shows that the deviation exists, respectively counting the ratio of the target criminal referee documents with different deviation types to the total amount of the target criminal referee documents according to the deviation types.
7. A method of testing the results of a law firm official according to claim 1, wherein said target criminal official document is plural and has a corresponding classification type, the method further comprising:
and respectively counting the ratio of the target criminal referee documents corresponding to different verification results under different classification types to the total amount of the target criminal referee documents according to the classification types.
8. A device for checking the result of a judge of a legal document, comprising:
a document determination unit for determining a target criminal referee document;
a content acquisition unit for acquiring target document contents from the target criminal referee document, wherein the target document contents are contents capable of influencing the referee result;
the judging result acquisition unit is used for inputting the target document content into a pre-constructed neural network classification model so as to obtain a judging result which is output by the neural network classification model and is adaptive to the target document content;
and the judging result checking unit is used for comparing the judging result output by the neural network classification model with the judging result in the target criminal judging document to obtain a checking result.
9. A storage medium having a program stored thereon, wherein the program, when executed by a processor, implements a method of verifying a result of a official document of a legal instrument as claimed in any one of claims 1 to 7.
10. An electronic device comprising at least one processor, and at least one memory, bus connected to the processor; the processor and the memory complete mutual communication through the bus; the processor is used for calling the program instructions in the memory to execute the method for checking the result of the legal document judge according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910887589.4A CN112613294A (en) | 2019-09-19 | 2019-09-19 | Method and device for inspecting judgment result of legal document |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910887589.4A CN112613294A (en) | 2019-09-19 | 2019-09-19 | Method and device for inspecting judgment result of legal document |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112613294A true CN112613294A (en) | 2021-04-06 |
Family
ID=75224235
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910887589.4A Pending CN112613294A (en) | 2019-09-19 | 2019-09-19 | Method and device for inspecting judgment result of legal document |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112613294A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107918921A (en) * | 2017-11-21 | 2018-04-17 | 南京擎盾信息科技有限公司 | Criminal case court verdict measure and system |
| US20180137350A1 (en) * | 2016-11-14 | 2018-05-17 | Kodak Alaris Inc. | System and method of character recognition using fully convolutional neural networks with attention |
| CN109308355A (en) * | 2018-09-17 | 2019-02-05 | 清华大学 | Legal decision prediction of result method and device |
| CN109376963A (en) * | 2018-12-10 | 2019-02-22 | 杭州世平信息科技有限公司 | A kind of criminal case charge law article unified prediction neural network based |
| CN110188192A (en) * | 2019-04-16 | 2019-08-30 | 西安电子科技大学 | A kind of multitask network struction and multiple dimensioned charge law article unified prediction |
-
2019
- 2019-09-19 CN CN201910887589.4A patent/CN112613294A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180137350A1 (en) * | 2016-11-14 | 2018-05-17 | Kodak Alaris Inc. | System and method of character recognition using fully convolutional neural networks with attention |
| CN107918921A (en) * | 2017-11-21 | 2018-04-17 | 南京擎盾信息科技有限公司 | Criminal case court verdict measure and system |
| CN109308355A (en) * | 2018-09-17 | 2019-02-05 | 清华大学 | Legal decision prediction of result method and device |
| CN109376963A (en) * | 2018-12-10 | 2019-02-22 | 杭州世平信息科技有限公司 | A kind of criminal case charge law article unified prediction neural network based |
| CN110188192A (en) * | 2019-04-16 | 2019-08-30 | 西安电子科技大学 | A kind of multitask network struction and multiple dimensioned charge law article unified prediction |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109873812B (en) | Anomaly detection method and device and computer equipment | |
| CN110276066B (en) | Entity association relation analysis method and related device | |
| CN110377730B (en) | Case-by-case classification method, apparatus, computer device, and storage medium | |
| CN109872162B (en) | Wind control classification and identification method and system for processing user complaint information | |
| EP2657884A2 (en) | Identifying multimedia objects based on multimedia fingerprint | |
| CN112860841B (en) | Text emotion analysis method, device, equipment and storage medium | |
| CN110909784B (en) | Training method and device of image recognition model and electronic equipment | |
| CN111797629B (en) | Method and device for processing medical text data, computer equipment and storage medium | |
| CN108959474B (en) | Entity relation extraction method | |
| CN110502677B (en) | Equipment identification method, device and equipment, and storage medium | |
| WO2020063524A1 (en) | Method and system for determining legal instrument | |
| CN110968689A (en) | Training method of criminal name and law bar prediction model and criminal name and law bar prediction method | |
| CN112818162A (en) | Image retrieval method, image retrieval device, storage medium and electronic equipment | |
| CN110490304B (en) | Data processing method and device | |
| KR20220116110A (en) | Method for determining a confidence level of inference data produced by artificial neural network | |
| CN111767390A (en) | Skill word evaluation method and device, electronic equipment and computer readable medium | |
| CN114372532A (en) | Method, device, equipment, medium and product for determining label marking quality | |
| WO2021051957A1 (en) | Judicial text recognition method, text recognition model obtaining method, and related device | |
| CN111552810B (en) | Entity extraction and classification method, entity extraction and classification device, computer equipment and storage medium | |
| CN112199500A (en) | Emotional tendency identification method and device for comments and electronic equipment | |
| CN114254588B (en) | Data tag processing method and device | |
| WO2022237065A1 (en) | Classification model training method, video classification method, and related device | |
| CN112613294A (en) | Method and device for inspecting judgment result of legal document | |
| CN114065858A (en) | Model training method and device, electronic equipment and storage medium | |
| US11514311B2 (en) | Automated data slicing based on an artificial neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |