CN114860900A - Sentencing prediction method and device - Google Patents

Abstract

The present application discloses a sentencing prediction method and device, which are used to solve the problems of incomplete and accurate judgment results in the prior art. The method proposed in this application includes: obtaining case-related information and crime fact description text; performing vectorization processing on word segments included in multiple chapters of the crime fact description text to obtain a first word vector corresponding to each segment Perform vectorization processing to obtain the second word vector corresponding to each word segment; perform feature extraction on the first word vector included in multiple chapters to obtain the first feature vector of each chapter in the multiple chapters. The feature vector determines the predicted category of each chapter; perform feature extraction on the second word vector included in the case-related information to obtain the second feature vector of the case-related information; according to the predicted categories of multiple first feature vectors corresponding to multiple chapters And the second eigenvector for law prediction, crime prediction and sentence prediction.

Description

Sentencing prediction method and device

technical field

The present application relates to the field of information technology, and in particular, to a sentencing prediction method and device.

Background technique

At present, the prediction of legal sentence mainly involves three sub-tasks: the prediction of the crime, the law and the sentence. The prediction process of the sentence is very complicated, not only the basic state of the defendant and the process of committing the crime, but also the relevant factors such as whether the defendant actively pleads guilty and surrenders. First, the existing technical solution lacks external information and only pays attention to the description of the criminal facts of the case. The description of criminal facts does play an irreplaceable role in predicting crimes and laws, but in the process of predicting the sentence, it is not realistic to rely only on the information in the description of criminal facts. Secondly, in the prior art solution, the relevant information corresponding to the three sub-tasks is extracted from the description of the crime facts, and the prediction is made according to the information corresponding to a single sub-task, without considering the relationship between the sub-tasks, resulting in the discrimination result. There are certain errors in comprehensiveness and accuracy.

SUMMARY OF THE INVENTION

The embodiments of the present application provide a sentencing prediction method and device, which are used to solve the problems of incomplete and accurate judgment results in the prior art.

In the first aspect, the embodiments of the present application provide a method for predicting sentencing, including:

Obtain case-related information and a description text of the crime facts, the case-related information includes at least one of witnesses, physical evidence, defendant information, testimony, the suspect's confession, and transcripts; the crime fact description text includes multiple chapters, and the Each of the plurality of chapters includes a plurality of clauses, and each of the plurality of sentences includes a plurality of participles;

performing vectorization processing on the word segments included in the plurality of chapters to obtain a first word vector corresponding to each word segment, and performing vectorization processing on the word segments included in the case-related information to obtain a second word vector corresponding to each segment; Perform feature extraction on the first word vectors included in the multiple chapters to obtain the first feature vector of each chapter in the multiple chapters, and determine the predicted category of each chapter according to the first feature vector of each chapter, so The prediction category mentioned above is the category of the law or the category of the crime or the category of the sentence;

Perform feature extraction on the second word vector included in the case-related information to obtain a second feature vector of the case-related information; according to the predicted categories of the multiple first feature vectors corresponding to the multiple chapters and the second feature The vector is used for law prediction, crime prediction and sentence prediction.

Based on the above scheme, the relevant information of the case is introduced on the basis of the description text of the crime facts to jointly predict the sentencing, which improves the prediction effect. At the same time, due to the complexity of the sentence, a topology structure is introduced in the multi-task, first predicting the crime, then predicting the law, and finally predicting the sentence on the basis of the crime and the law, which further improves the effect of sentence prediction.

In a possible implementation manner, performing feature extraction on the first word vectors included in the multiple chapters to obtain the first feature vector of each chapter in the multiple chapters, including: pairing based on the first word vector included in the first chapter. The word segmentation in the first chapter is filtered to obtain the filtered first chapter, the first word vectors in the multiple clauses included in the filtered first chapter are all related to sentencing prediction, and the first chapter is the Describe any one of the multiple chapters; combine the multiple clauses included in the filtered first chapter to obtain multiple clause combinations, and each clause combination in the multiple clause combinations includes at least two clauses clause;

The first semantic vector encoder is used to extract features for each clause combination to obtain word-level feature vectors of each clause combination; perform feature splicing on word-level feature vectors of multiple clause combinations to obtain each segment The sentence vector representation of the sentence combination; the feature extraction is performed on the sentence vector of each sentence combination by the second semantic vector encoder to obtain the sentence level feature vector of each sentence combination; The feature vector performs feature splicing to obtain the first feature vector.

Based on the above scheme, through feature extraction, contextual features between multiple word segments and contextual features between clauses in each chapter are obtained.

In a possible implementation manner, the method further includes: encoding a position vector corresponding to a plurality of first word vectors included in the filtered first chapter, where the position vector corresponding to the first word vector is used to represent the The position of the word segment corresponding to the first word vector in the text corresponding to the first chapter; the first word vector of the multiple word segments included in the filtered first chapter and the corresponding position vector are fused to obtain the first word vector in the first chapter. The fusion word vector of multiple word segmentations;

The feature extraction is performed on each clause combination by the first semantic vector encoder to obtain a word-level feature vector of each clause combination, including: according to the fusion word vector of a plurality of segmented words included in the first clause combination, using The first semantic vector encoder performs feature extraction on the first clause combination to obtain a word-level feature vector of the first clause combination, where the first clause combination is any one of the multiple clause combinations Clause combination.

Based on the above solution, by fusing the first word vector and the position vector, the relative and absolute position information of each word segment in the sentence can be obtained in the subsequent encoding process.

In a possible implementation manner, the method further includes: encoding a position vector corresponding to a plurality of first sentence vectors included in the filtered first chapter, where the position vector corresponding to the first sentence vector is used to represent the The position of the clause corresponding to the first sentence vector in the text corresponding to the first chapter; the plurality of first sentence vectors included in the filtered first chapter and the corresponding position vector are fused to obtain the multiplication in the first chapter. The fused sentence vector of each clause;

The feature extraction is performed on the sentence vector of each clause combination by the second semantic vector encoder to obtain the clause-level feature vector of each clause combination, including: according to the plurality of clauses included in the first clause combination The fusion sentence vector adopts the second semantic vector encoder to perform feature extraction on the first clause combination, so as to obtain the clause-level feature vector of the first clause combination, and the first clause combination is the Any one of the clause combinations.

Based on the above solution, by fusing the first sentence vector and the position vector, the relative and absolute position information of each sentence in the sentence combination can be obtained in the subsequent encoding process.

In a possible implementation, the first semantic vector encoder includes N attention network layers, a first neural network layer, and a first single-head attention layer; each attention layer in the N attention network layers The force network layer includes the first multi-head attention layer and the first addition normalization layer; N is a positive integer;

The first semantic vector encoder is used to perform feature extraction on the first clause combination according to the fusion word vector of the plurality of word segments included in the first clause combination, so as to obtain the word-level feature vector of the first clause combination, including :

The plurality of attention modules included in the first multi-head attention layer in the i-th attention network layer respectively perform attention operations on the fusion word vectors of the plurality of word segmentations included in the first sentence combination, to obtain the outputs of the multiple attention modules;

The first addition normalization layer in the i-th attention network layer splices the output results of the multiple attention modules to obtain a splicing result; according to the output results of the i-1-th attention network layer Perform linear transformation on the splicing result to obtain the first output result of the first multi-head attention layer of the ith attention network layer; i is a positive integer less than or equal to N; for the ith attention network The first output result of the first multi-head attention layer of the layer is normalized to obtain a second output result, and the second output result is used for the linear transformation of the i+1th attention network layer;

Perform feature extraction on the second output result of the Nth attention network layer through the first neural network layer to obtain the feature matrix of the first word segmentation of each sentence combination;

The feature information in the feature matrix of the first word segmentation is extracted through the first single-head attention layer, and the word-level feature vector of the first sentence combination is obtained.

Based on the above scheme, N attention network layers are used to perform feature extraction on multiple word segmentations included in the first sentence combination, which can capture the long-distance features between word segmentation and word segmentation in the text, and can extract rich contextual semantic representation information. , to enhance the feature extraction ability.

In a possible implementation, the second semantic vector encoder includes N attention network layers, a second neural network layer and a second single-head attention layer; each attention network layer in the N attention network layers The force network layer includes a second multi-head attention layer and a second addition normalization layer; N is a positive integer;

The second semantic vector encoder is used to perform feature extraction on the first clause combination according to the fusion sentence vector of the plurality of clauses included in the first clause combination, so as to obtain the clause-level feature vector of the first clause combination. ,include:

The plurality of attention modules included in the second multi-head attention layer in the i-th attention network layer respectively perform attention calculation on the fused sentence vectors of the plurality of sentences included in the first sentence combination, to obtain obtain the outputs of the multiple attention modules;

The second addition normalization layer in the i-th attention network layer splices the output results of the multiple attention modules to obtain a splicing result; according to the output results of the i-1-th attention network layer Perform linear transformation on the splicing result to obtain the third output result of the second multi-head attention layer of the ith attention network layer; i is a positive integer less than or equal to N; for the ith attention network The third output result of the second multi-head attention layer of the layer is normalized to obtain a fourth output result, and the fourth output result is used for the linear transformation of the i+1th attention network layer;

Perform feature extraction on the fourth output result of the Nth attention network layer through the second neural network layer to obtain a feature matrix of the first clause of each clause combination;

The feature information in the feature matrix of the first clause is extracted through the second single-head attention layer, and the clause-level feature vector of the first clause combination is obtained.

Based on the above scheme, N attention network layers are used to perform feature extraction on multiple clauses included in the first clause combination, which can capture long-distance features between clauses in the text and extract rich context. Semantically represent information and enhance the ability to extract features.

In a possible implementation manner, according to the prediction categories of the plurality of first feature vectors corresponding to the plurality of chapters and the second feature vectors, the law prediction, the crime prediction and the sentence prediction are performed, including:

Perform nonlinear transformation on the second eigenvector and the first eigenvectors corresponding to the plurality of chapters and the first eigenvector whose prediction category is the law category to obtain a law prediction vector, and obtain a law prediction vector according to the law prediction vector make legal predictions;

Perform nonlinear transformation on the second eigenvector, the first eigenvectors corresponding to the plurality of chapters and the first eigenvector whose predicted category is the crime category, and the ordinance prediction vector to obtain the crime prediction vector. Describe the crime prediction vector for crime prediction;

Perform nonlinear transformation on the second feature vector, the first feature vector corresponding to the multiple chapters and the first feature vector whose prediction category is the sentence category, the legal article prediction vector, and the crime prediction vector to obtain A sentence prediction vector, and a sentence prediction is performed according to the sentence prediction vector.

Based on the above scheme, the topological structure is used in the sentencing prediction, firstly predicting the crime, then predicting the law, and finally predicting the sentence on the basis of the crime and the law, which further improves the effect of sentence prediction.

In a possible implementation manner, the case-related information includes first data and second data; wherein, the first data includes at least one of testimony, a suspect's confession, and a transcript, and the second data includes a person at least one of evidence, physical evidence, and defendant information; performing vectorization processing on the word segments included in the case-related information to obtain a second word vector corresponding to each word segment, including:

Perform vectorization processing on the word segmentation included in the first data to obtain a second word vector corresponding to each word segmentation in the first data;

Determine the category to which each participle included in the second data belongs, and determine a category vector corresponding to the category to which each participle included in the second data belongs from the data vector table; the data vector table includes a plurality of categories corresponding to Category vector; the second word vector corresponding to each word segmentation is determined from the category vector corresponding to the category to which each word segment included in the second data belongs.

Based on the above scheme, the case-related information is introduced on the basis of the description text of the crime facts, and the feature extraction of the relevant information of the case is carried out.

In a possible implementation manner, the filtered first chapter is obtained by filtering the word segmentation in the first chapter based on the first word vector included in the first chapter, including:

The plurality of first word vectors included in the first chapter are filtered through a convolutional neural network to obtain the filtered first chapter.

In a second aspect, an embodiment of the present application provides a sentencing prediction device, including an acquisition unit and a processing unit;

The obtaining unit is used to obtain case-related information and a description text of the crime facts, the case-related information includes at least one of witness evidence, physical evidence, defendant information, testimony, the suspect's personal testimony, and transcripts; the crime fact description text Including a plurality of articles, each article in the plurality of articles includes a plurality of clauses, and each clause in the plurality of sentences includes a plurality of participles;

The processing unit is configured to perform vectorization processing on the word segments included in the plurality of chapters to obtain a first word vector corresponding to each word segment, and perform vectorization processing on the word segments included in the case-related information to obtain each word segment The corresponding second word vector; perform feature extraction on the first word vector included in the multiple chapters to obtain the first feature vector of each chapter in the multiple chapters, and determine the each chapter according to the first feature vector of each chapter. The prediction category of each chapter, the prediction category is the category of the law or the category of the crime or the category of the sentence;

The processing unit is further configured to perform feature extraction on the second word vector included in the case-related information to obtain a second feature vector of the case-related information; The predicted category and the second feature vector are used to predict legal articles, crimes and sentence predictions.

In a possible implementation manner, when the processing unit performs feature extraction on the first word vectors included in the multiple chapters to obtain the first feature vector of each chapter in the multiple chapters, the processing unit is specifically used for:

Based on the first word vector included in the first chapter, the word segmentation in the first chapter is filtered to obtain the filtered first chapter, and the first word vectors in the multiple clauses included in the filtered first chapter are all the same as Sentencing prediction is related, and the first chapter is any one of the multiple chapters;

Combining the plurality of clauses included in the filtered first chapter to obtain a plurality of clause combinations, each clause combination in the plurality of clause combinations includes at least two clauses;

Perform feature extraction on each clause combination by the first semantic vector encoder to obtain a word-level feature vector of each clause combination;

Feature splicing is performed on the word-level feature vectors of multiple clause combinations to obtain the sentence vector representation of each clause combination;

Perform feature extraction through the sentence vector of each clause combination by the second semantic vector encoder to obtain the clause-level feature vector of each clause combination;

Feature splicing is performed on the clause-level feature vectors of multiple clause combinations to obtain the first feature vector.

In a possible implementation manner, the processing unit is further configured to: encode a position vector corresponding to a plurality of first word vectors included in the filtered first chapter, where the position vector corresponding to the first word vector is used to represent The position of the word segment corresponding to the first word vector in the text corresponding to the first chapter; the first word vector of the plurality of word segments included in the filtered first chapter and the corresponding position vector are fused to obtain the first word vector. The fusion word vector of multiple word segmentations in the chapter;

The processing unit, when performing feature extraction on each clause combination by the first semantic vector encoder, to obtain the word-level feature vector of each clause combination, is specifically used for: according to the plurality of clauses included in the first clause combination. The fusion word vector of each word segment adopts the first semantic vector encoder to perform feature extraction on the first clause combination, so as to obtain the word-level feature vector of the first clause combination, and the first clause combination is the plurality of Any one of the clause combinations.

In a possible implementation manner, the processing unit is further configured to: encode a position vector corresponding to a plurality of first sentence vectors included in the filtered first chapter, where the position vector corresponding to the first sentence vector is used to represent The position of the clause corresponding to the first sentence vector in the text corresponding to the first chapter; the first chapter is obtained by fusing a plurality of first sentence vectors included in the filtered first chapter with the corresponding position vector The fused sentence vector of multiple clauses in ;

The processing unit, when performing feature extraction on the sentence vector of each clause combination by the second semantic vector encoder to obtain the clause-level feature vector of each clause combination, is specifically used for: according to the first clause The fusion sentence vector of the multiple clauses included in the combination uses the second semantic vector encoder to perform feature extraction on the first clause combination, so as to obtain the clause-level feature vector of the first clause combination. The combination is any one of the multiple clause combinations.

In a possible implementation, the first semantic vector encoder includes N attention network layers, a first neural network layer, and a first single-head attention layer; each attention layer in the N attention network layers The force network layer includes the first multi-head attention layer and the first addition normalization layer; N is a positive integer;

The processing unit uses the first semantic vector encoder to perform feature extraction on the first clause combination according to the fusion word vector of the plurality of word segments included in the first clause combination, so as to obtain the word level of the first clause combination. When the feature vector is used, it is specifically used for: the multiple attention modules included in the first multi-head attention layer in the i-th attention network layer respectively fuse the multiple word segmentation included in the first sentence combination. The vector performs attention operation to obtain the outputs of the multiple attention modules; the first addition normalization layer in the ith attention network layer splices the output results of the multiple attention modules , obtain the splicing result; perform linear transformation on the splicing result according to the output result of the i-1th attention network layer, and obtain the first output result of the first multi-head attention layer of the ith attention network layer; i is a positive integer less than or equal to N; the first output result of the first multi-head attention layer of the i-th attention network layer is normalized to obtain the second output result, and the second output result is The linear transformation of the i+1th attention network layer; the feature extraction is performed on the second output result of the Nth attention network layer through the first neural network layer, and the first result of each sentence combination is obtained. The feature matrix of the word segmentation; the feature information in the feature matrix of the first word segmentation is extracted through the first single-head attention layer, and the word-level feature vector of the first sentence combination is obtained.

In a possible implementation, the second semantic vector encoder includes N attention network layers, a second neural network layer and a second single-head attention layer; each attention network layer in the N attention network layers The force network layer includes a second multi-head attention layer and a second addition normalization layer; N is a positive integer;

The processing unit uses the second semantic vector encoder to perform feature extraction on the first clause combination according to the fusion sentence vector of the plurality of clauses included in the first clause combination, so as to obtain the score of the first clause combination. When the sentence-level feature vector is used, it is specifically used for: the multiple attention modules included in the second multi-head attention layer in the i-th attention network layer respectively combine the multiple clauses included in the first clause The fused sentence vector is used for attention calculation to obtain the output of the multiple attention modules; the second addition normalization layer in the i-th attention network layer is used for the output of the multiple attention modules. The results are spliced to obtain a splicing result; the splicing result is linearly transformed according to the output result of the i-1 th attention network layer, and the third output of the second multi-head attention layer of the ith attention network layer is obtained. Result: i is a positive integer less than or equal to N; the third output result of the second multi-head attention layer of the i-th attention network layer is normalized to obtain a fourth output result. The output result is used for the linear transformation of the i+1th attention network layer; the feature extraction is performed on the fourth output result of the Nth attention network layer through the second neural network layer, and each sentence combination is obtained. The feature matrix of the first clause; extract the feature information in the feature matrix of the first clause through the second single-head attention layer, and obtain the clause-level feature vector of the first clause combination.

In a possible implementation manner, the processing unit, when performing the prediction of laws, charges and sentences according to the prediction categories of the plurality of first feature vectors corresponding to the plurality of chapters and the second feature vectors, It is specifically used for: performing nonlinear transformation on the second feature vector and the first feature vectors corresponding to the multiple chapters and the first feature vector whose prediction category is the law category to obtain the law prediction vector, and according to the Perform legal prediction on the legal article prediction vector; perform non-inference on the second feature vector, the first feature vector whose predicted category is the crime category in the second feature vector, the plurality of first feature vectors corresponding to the multiple chapters, and the legal article prediction vector. Linearly transforming to obtain a crime prediction vector, and performing crime prediction according to the crime prediction vector; for the second feature vector and the first feature vectors corresponding to the multiple chapters, the predicted category is the first feature vector of the sentence category, The said law prediction vector and the said crime prediction vector are nonlinearly transformed to obtain a sentence prediction vector, and the sentence prediction is performed according to the said sentence prediction vector.

In a possible implementation manner, the case-related information includes first data and second data; wherein, the first data includes at least one of testimony, a suspect's confession, and a transcript, and the second data includes a person At least one of evidence, physical evidence, and defendant information; the processing unit, when performing vectorization processing on the word segments included in the case-related information to obtain the second word vector corresponding to each word segment, is specifically used for: The word segmentation included in the first data is vectorized to obtain a second word vector corresponding to each word segmentation in the first data; the category to which each word segmentation included in the second data belongs is determined, and the The category vector corresponding to the category to which each participle included in the second data belongs; the data vector table includes category vectors corresponding to multiple categories; the category vector corresponding to the category to which each participle included in the second data belongs is determined Describe the second word vector corresponding to each participle.

In a possible implementation manner, the processing unit, when performing filtering processing on the word segmentation in the first chapter based on the first word vector included in the first chapter to obtain the filtered first chapter, is specifically used to: pass the volume The product neural network performs filtering processing on the plurality of first word vectors included in the first chapter to obtain the filtered first chapter.

In a third aspect, an embodiment of the present application provides a sentencing prediction device, including a memory and a processor;

the memory for storing program instructions;

The processor is configured to invoke the program instructions stored in the memory, and execute the method described in the first aspect and any possible implementation manner included in the first aspect according to the obtained program.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where computer instructions are stored in the computer-readable storage medium, and when the computer instructions are executed on a computer, the computer can execute the first aspect and the first The method described in any of the possible implementations included in the aspect.

In addition, for the technical effects brought by any one of the implementations of the second aspect to the fourth aspect, reference may be made to the technical effects brought by the first aspect and different implementations of the first aspect, which will not be repeated here.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are For some embodiments of the present application, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

1A is a schematic diagram of a system architecture provided by an embodiment of the present application;

FIG. 1B is a schematic structural diagram of a server provided by an embodiment of the present application;

2 is a schematic flowchart of a sentencing prediction method provided by an embodiment of the present application;

3 is a schematic diagram of a network model for feature extraction of case-related information provided by an embodiment of the present application;

FIG. 4A is a schematic diagram of a network model for feature extraction of a crime fact description text provided by an embodiment of the present application;

FIG. 4B is a schematic flowchart of feature extraction of a crime fact description text provided by an embodiment of the present application;

5 is a schematic flowchart of obtaining a word-level feature vector according to an embodiment of the present application;

FIG. 6 is a schematic diagram of an attention network layer provided by an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a classifier according to an embodiment of the present application;

8 is a schematic structural diagram of a sentencing prediction model provided by an embodiment of the present application;

9 is a schematic diagram of a sentencing prediction device provided by an embodiment of the present application;

FIG. 10 is a schematic diagram of another sentencing prediction apparatus provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations.

Thus, the following detailed description of the embodiments of the application provided in the accompanying drawings is not intended to limit the scope of the application as claimed, but is merely representative of selected embodiments of the application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

It should be noted that relational terms such as the terms "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any relationship between these entities or operations. any such actual relationship or sequence exists. Moreover, the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device that includes a list of elements includes not only those elements, but also includes not explicitly listed or other elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

Aiming at the problem in the prior art that the sentence is only predicted based on the description of the criminal facts of the case, resulting in a certain error in the comprehensiveness and accuracy of the judgment result, the present application provides a sentencing prediction method. On the basis, the relevant information of the case is added. By encoding and extracting two kinds of information, the description text of the crime fact and the relevant information of the case, the sentencing prediction can be realized, and the accuracy of the sentencing prediction can be improved.

FIG. 1A exemplarily shows a system architecture to which the embodiments of the present application are applied, and the system architecture may include a sentencing prediction apparatus. In some embodiments, the sentencing prediction apparatus may include one or more servers 100 , three servers are taken as an example in FIG. 1A . The server 100 may be implemented by a physical server or a virtual server. The server can be implemented by a single server or by a server cluster composed of multiple servers. A single server or server cluster is used to implement the sentencing prediction method provided by this application. Optionally, the server 100 may be connected to the terminal device, and receive the sentencing prediction task sent by the terminal device, or send the sentencing prediction result to the terminal device. For example, the terminal device may be a mobile phone, a tablet computer, a personal computer, and the like.

As an example, referring to FIG. 1B , the server may include a processor 110 , a communication interface 120 and a memory 130 . Of course, the server 100 may also include other components, which are not shown in FIG. 1B .

Taking the connection of the server 100 with multiple terminal devices as an example, the communication interface 120 is used for communicating with different terminal devices, for receiving sentencing prediction tasks sent by the terminal devices, or sending sentencing prediction results to the terminal devices.

In this embodiment of the present application, the processor 110 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, which may implement Alternatively, each method, step, and logic block diagram disclosed in the embodiments of the present application are executed. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the methods disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

The processor 110 is the control center of the server 100, using various interfaces and routes to connect various parts of the entire server 100, by running or executing the software programs/or modules stored in the memory 130, and calling the data stored in the memory 130, Various functions of the server 100 are executed and data is processed. Optionally, processor 110 may include one or more processing units. The processor 110, for example, may be a control component such as a processor, a microprocessor, a controller, etc., for example, a general-purpose central processing unit (central processing unit, CPU), a general-purpose processor, a digital signal processing (digital signal processing, DSP), a dedicated Integrated circuits (application specific integrated circuits, ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

The memory 130 may be used to store software programs and modules, and the processor 110 executes various functional applications and data processing by running the software programs and modules stored in the memory 130 . The memory 130 may mainly include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required for at least one function, and the like; the stored data area may store data created according to business processing, and the like. As a non-volatile computer-readable storage medium, the memory 130 can be used to store non-volatile software programs, non-volatile computer-executable programs and modules. The memory 130 may include at least one type of storage medium, for example, may include a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a Programmable Read Only Memory (PROM), Read Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Magnetic Memory, Magnetic Disk, Optical Disk and many more. The memory 130 is, but is not limited to, any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The memory 130 in this embodiment of the present application may also be a circuit or any other device capable of implementing a storage function, for storing program instructions and/or data.

It should be noted that the structures shown in FIG. 1A and FIG. 1B above are only examples, which are not limited in this embodiment of the present invention.

In some scenarios, the sentencing prediction method provided by the embodiments of the present application may be implemented by one or more local terminal devices.

An embodiment of the present application provides a sentencing prediction method, and FIG. 2 exemplarily shows a flow of the sentencing prediction method, and the flow can be executed by a sentencing prediction device, which can be located in the server 100 shown in FIG. 1B , for example, it can be It is the processor 110, and may also be the server 100. In the subsequent description, the server 100 is used as an example for description. For the convenience of description, the subsequent description of the server 100 will not exemplify the numerical designation. The sentencing prediction device may also be located in a local terminal device. The specific process is as follows:

201. Obtain case-related information and the description text of the crime facts.

In some embodiments, the case-related information refers to other auxiliary information related to the case, such as witness evidence, physical evidence, defendant information, testimony, suspect testimony, and transcripts. The descriptive text of the crime facts may include defendant information and crime information. The defendant's information may include information on the defendant's previous convictions, such as the number of criminal convictions, fines for criminal convictions, political deprivation for criminal convictions, sentence for criminal convictions, and charges for criminal convictions. The crime information may include a description of the crime facts, or a description of the crime facts and the amount of money involved in the case.

In some embodiments, after the crime fact description text is obtained, the crime fact description text is segmented. The crime fact description text may be segmented based on the tokenizer, which is not specifically limited in this application. For example, the tokenizer may use a text analysis tokenizer Analysis, a Chinese language processing package (Han Language Processing, HanLP).

In some embodiments, after the crime fact description text is segmented, the crime fact description text may be divided into multiple chapters with a fixed number of word segments or clauses, each chapter includes multiple clauses, and each clause includes multiple clauses. a participle. The crime fact description text may also be divided into multiple chapters according to paragraphs, which is not specifically limited in this embodiment of the present application.

In some embodiments, the descriptive text of the crime facts and the case-related information may come from a terminal device connected to the server. In some embodiments, a user can trigger a sentencing prediction task through a terminal device. When sending a sentencing prediction task to the server, the description text of the crime facts and case-related information can be sent to the server, and the sentencing prediction task is used to instruct the server to perform sentencing prediction. After the server receives the sentencing prediction task, it executes the method flow of sentencing prediction.

202. Perform vectorization processing on the word segments included in the multiple chapters to obtain a first word vector corresponding to each word segment, and perform vectorization processing on the word segments included in the case-related information to obtain a second word vector corresponding to each word segment.

In some embodiments, the crime fact description text is written and official, and has the characteristics of being concise and comprehensive. Therefore, after determining multiple chapters included in the crime fact description text, perform vectorization processing on multiple word segments corresponding to the multiple chapters, and obtain: The word vector corresponding to each participle. For the convenience of description, the word vector corresponding to the segmented words included in the chapter is called the first word vector. In some scenarios, the crime fact description text is first processed at word granularity, and the crime fact description text is converted into numerical form. Specifically, taking the first chapter as an example, each participle included in the first chapter corresponds to a token, and the first chapter can be expressed as T=(t ₁ ,t ₂ ,...,t _n ). Among them, T represents the token sequence, t _i represents a token of a token, and n is the number of tokens included in the first chapter. Further, the token sequences included in the first chapter can be mapped to integer sequences. The sequence of integers can be expressed as D(t→d)=(d ₁ , d ₂ , . . . , d _n ), where n is the number of participles included in the first chapter. For example, the correspondence between the token corresponding to each participle and an integer value can be determined through a dictionary. For example, the dictionary stores the correspondence between the tokens corresponding to all the word segmentations encountered in the current scene and an integer value. For example, training annotations based on statistical features can obtain 5,000 word segments, and the dictionary stores the correspondence between tokens and integer values corresponding to 5,000 word segments. Different participles correspond to different integer values.

In some embodiments, after the crime fact description text is converted into a numerical form in the form of text at word granularity, the word segmentation may be mapped to a vector space to obtain multiple word vectors. A word vector sequence is stored in the sentencing prediction device, and the number of the word vector sequence is the size of the dictionary, and a plurality of first word vectors are mapped according to the word vector sequence through the numerical values corresponding to the plurality of word segmentations. Specifically, the word vector sequence can be expressed as E=(e ₁ ,e ₂ ,...,e _s ), where s is the number of segmented words included in the dictionary, e _i is the word vector, and the length can be set, e _i ∈R ^k , k is the length of the word vector. For example, if a word vector in the dictionary is "valid", the length of the word vector is 2. Each word segment included in the crime fact description text corresponds to a token, and the value corresponding to each token can be mapped to a vector through E, and all the mapped word vectors are in the original order of the token, that is, the word segmentation in the crime fact description text. Combined sequentially, the first word vector of each chapter in the crime description text including the word segmentation is obtained. As an example, taking the first chapter including s word segments as an example, after the word vector sequence E is mapped, the first chapter can be expressed as X=(x ₁ ,x ₂ ,...,x _s ), where x _i is The first word vector corresponding to the ith participle, x _i ∈ R ^k , where k is the length of the first word vector. In some scenarios, the sequence of word vectors can be generated by the sentencing device model. After the sentencing prediction device receives the sentencing prediction task sent by the terminal device, it can automatically generate a word vector sequence. After converting the word segmentation of the crime fact description text into numerical values, the word segmentation can be mapped to the vector space according to the word vector sequence to obtain multiple word vectors.

In other embodiments, after obtaining the case-related information, the case-related information includes the first data and the second data. It can be understood that the first data and the second data may also be referred to by other names, for example, the first data may be referred to as continuous data, and the second data may be referred to as discrete data. Wherein, the first data may include at least one item of testimony, a suspect's oral testimony, and a transcript, and the second data may include at least one item of witness testimony, physical evidence, and defendant information.

In some scenarios, the word segment included in the first data may be vectorized to directly obtain a second word vector corresponding to each word segment included in the first data. The method for vectorizing the word segments included in the first data is the same as the above-mentioned method for vectorizing the word segments included in the crime fact description text, and details are not described herein again.

In other scenarios, the multiple word segments included in the first data may belong to different categories. Of course, some word segments in the first data may belong to the same category, that is, each category includes multiple word segments. The vectorized value ranges corresponding to multiple categories may be different, and the range of the vectorized value ranges corresponding to multiple categories is relatively large. Therefore, after the word segmentation included in the first data is vectorized, a normalization operation can be performed on word vectors obtained by vectorizing multiple word segmentations included in the first data to obtain a second word vector corresponding to each word segmentation . Therefore, the second word vector corresponding to the ith participle can satisfy the conditions shown in the following formula (1):

Among them, c′ _i represents the second word vector corresponding to the ith participle in the first data, μ _c represents the mean of all the values included in the first data, σ _c represents the variance, and c _i represents the ith participle to be vectorized the resulting word vector.

In some scenarios, the first data including the first word vector of the category to which all the segmented words belong may be represented by a vector sequence, for example, represented as C′=(c′ ₁ ,c′ ₂ ,c′ ₃ ,...,c′ _g ). Among them, C′∈R ^g , and g is the number of categories of the first data.

In other embodiments, the category to which each word segment included in the second data belongs is determined, and a category vector corresponding to the category described in each word segment included in the second data is determined from the data vector table. The number of categories in the second data is less than the number of categories in the first data. The second word vector corresponding to each participle may be determined from a category vector corresponding to a category to which each participle included in the second data belongs by using a pre-constructed data vector table. The data vector table includes category vectors corresponding to multiple categories.

203. Perform feature extraction on the first word vectors included in the multiple chapters to obtain a first feature vector of each chapter in the multiple chapters, and determine the predicted category of each chapter according to the first feature vector of each chapter.

204. Perform feature extraction on the second word vector included in the case-related information to obtain a second feature vector of the case-related information.

For example, the second feature vector can be obtained by splicing the second word vector included in the case-related information by using the Concat function.

205 , according to the prediction categories of the plurality of first feature vectors corresponding to the plurality of chapters and the second feature vectors, perform law prediction, crime prediction, and sentence prediction.

In some embodiments, after obtaining the sentencing prediction result, the server may send the sentencing prediction result to the terminal device, and the user may obtain the sentencing prediction result through the terminal device.

Exemplarily, a sentencing prediction model may be deployed in the sentencing prediction device, and the sentencing prediction device executes steps 202-205 through the sentencing prediction model.

Through the above scheme, the case-related information is introduced on the basis of the descriptive text of the crime facts to jointly predict the sentencing, which improves the prediction effect. At the same time, due to the complexity of the sentence, a topology structure is introduced in the multi-task, first predicting the crime, then predicting the law, and finally predicting the sentence on the basis of the crime and the law, which further improves the effect of sentence prediction.

In a possible implementation manner, a feedforward neural network may be used when performing feature extraction on the second word vector included in the case-related information in step 204, as shown in FIG. 3 . For example, inputting the second word vector in the first data into the first feedforward neural network to obtain the output vector of the first data. For example, the first feedforward neural network includes a fully connected layer, and the second word vector in the first data is input into the fully connected layer to obtain an output vector of the first data. The output vector of the first data satisfies the conditions shown in the following formula (2):

T _c =Relu(C'W ^c +b _c ); (2)

Wherein, T _c is the output vector of the first data, T _c ∈ R ^p , p is the vector dimension of the output vector of the first data, and C′ is the vector sequence of the first data. Relu is the activation function, W ^c is the transformation matrix, and b _c is the bias term. Through the above formula, different categories of data that are isolated from each other can be fused together.

In some embodiments, the second word vector corresponding to the word segmentation included in the second data may be input into the second feedforward neural network to obtain an output vector of the second data, as shown in FIG. 3 . For example, inputting the second word vector in the second data into the second feedforward neural network to obtain the output vector of the second data. For example, the second feedforward neural network includes a fully connected layer, and the second word vector in the second data is input into the fully connected layer to obtain an output vector of each category included in the second data. The second data includes the output vector of the ith category that satisfies the conditions shown in the following formula (3):

T _di =Relu(d′ _i W ^di +b _di ); (3)

Among them, T _di represents the output vector corresponding to the ith category in the second data, Relu represents the activation function, W ^di represents the transformation matrix, and b _di represents the bias term. Further, the output vector of the second data can be represented as T _d =(T _d1 , T _d2 , . . . , T _dq ); wherein, q is the number of categories included in the second data.

Further, after obtaining the output vector of the first data and the output vector of the second data, the output vector of the first data and the output vector of the second data are spliced to obtain the second feature vector of the case-related information, as shown in the figure. 3 shown. The second eigenvector satisfies the conditions shown in the following formula (4):

T _m =Concat(T _c ,T _d ); (4)

Wherein, T _m represents the second feature vector, concat represents the concatenation function, T _c represents the output vector of the first data, and T _d represents the output vector of the second data.

In a possible implementation manner, when the first feature vector of each chapter is obtained in step 203, word-level encoding may be performed by the first semantic vector encoder, and then sentence-level encoding may be performed by the second semantic vector encoder level coding. The first semantic vector encoder may also be referred to as a word encoder, and the second semantic vector encoder may also be referred to as a sentence encoder. In some embodiments, before performing word-level encoding and sentence-level encoding, filtering is performed on the word segments included in each chapter to filter out word segments irrelevant to sentencing prediction. For example, when performing filtering, a neural network, such as a convolutional neural network, can be used.

As an example, as shown in FIG. 4A , a convolutional neural network is used for word segmentation filtering as an example. Taking the first chapter as an example, the word segmentation in the first chapter can be filtered based on the first word vector included in the first chapter to obtain the filtered first chapter. Word vectors are all related to sentencing prediction. Specifically, the filtered first chapter may be obtained by filtering the word segmentation included in the first chapter through a convolutional neural network. The advantage of a convolutional neural network over a fully connected neural network is that the convolutional neural network has fewer parameters and is faster to compute. Convolutional neural network performs text convolution. The convolution kernel in the convolutional neural network can be understood as a filter, similar to the high-pass filter in the communication field. By convolving the first word vector included in the first chapter, the Meaningful word vectors are passed through (ie, the first word vectors related to sentencing prediction), and meaningless words (such as "的", "計", etc.) are ignored. Using a convolutional neural network to filter the word segmentation included in the first chapter can make each word as a word with the characteristics of a word without overfitting. The convolution operation is performed by the convolution kernel, and finally the feature representation of the token sequence D corresponding to the first chapter is obtained, which can be expressed as C=(c ₁ ,c ₂ ,..., _cm ), as shown in Figure 4A. After filtering, the first word vector of the multiple word segments included in the first chapter has n-gram context features after convolution, that is, word segmentation and word segmentation are no longer isolated. For example, when n=2, the first word vector in the filtered first chapter has contextual features with the two word vectors before and after respectively.

In some embodiments, the crime fact description text belongs to the long chapter level, which inevitably leads to a long-distance dependency problem. In order to further obtain the relationship between word segmentation and word segmentation, the sentences in the text can be recombined. Specifically, multiple clauses included in the filtered first chapter may be combined to obtain multiple clause combinations, and each clause combination in the multiple clause combinations includes at least two clauses. The filtered first chapter can be represented as C"=( _ci ", _c2 ",..., _cm "). Exemplarily, when clauses are combined, the order between clauses is not limited. It can be understood that after multiple clauses are arranged in different orders, multiple clause combinations can be formed.

Referring to FIG. 4A , further, feature extraction is performed on each clause combination by the first semantic vector encoder, so as to obtain a word-level feature vector of each clause combination. Then, feature splicing is performed on the word-level feature vectors of multiple clause combinations to obtain the sentence vector representation of each clause combination. Then, perform feature extraction through the sentence vector of each clause combination by the second semantic vector encoder to obtain clause-level feature vector of each clause combination; perform feature splicing on the clause-level feature vectors of multiple clause combinations. to get the first eigenvector.

As an example, the first semantic vector encoder may use a Transformer encoder. The Transformer encoder performs feature extraction on each sentence combination through the attention mechanism, and has obtained the word-level feature vector of each sentence combination.

In some embodiments, when performing word-level encoding by the first semantic vector encoder, the position of each word segment may be combined. For example, as shown in FIG. 4B , the first word vectors of the multiple word segments included in the filtered first chapter are fused with the corresponding position vectors to obtain the fused word vector of the multiple word segments in the first chapter. The fusion word vector satisfies the conditions shown in the following formula (5):

代表元素对应相加操作符号。Wherein, C _p represents the fusion word vector, PE represents the position encoding vector, and C″ represents the first word vector corresponding to the multiple word segments included in the filtered first chapter.

Represents the element corresponding to the addition operator symbol.

In the above formula, the method of adding the corresponding elements is selected for the fusion of position coding vectors, rather than the method of vector splicing, so that there will be no problem of increasing parameters due to the method of vector splicing, and it is not easy to appear overfitting. .

Exemplarily, the position vector can be obtained by encoding a sine function and a cosine function. Specifically, after encoding the position vectors corresponding to the multiple first word vectors included in the first chapter after the combination, the position vector of the i-th word segment in the chapter satisfies the conditions shown in the following formulas (6) (7):

Among them, pos represents the position of the current word in the sentence, i represents the index of each word vector in the vector in the first chapter, and dmodel represents the dimension of the word vector. Therefore, given the position pos of a word segment, a position vector of dmodel dimension can be generated according to the above formulas (6) and (7). The generated position vector is the absolute position encoding, but because the trigonometric function is used, the relative position information between the word segmentations is also included in the absolute position encoding.

By performing position encoding on multiple first word vectors and fusing them with multiple first word vectors to obtain fused word vectors, it is possible to avoid being unable to know the position information of the word segmentation in the sentence, which will affect the final sentencing prediction result. It is understandable that, under normal circumstances, Transformer does not have a mechanism to capture the relative position of the word segmentation in the sentence, and the output will not be affected when the word segmentation is exchanged. When the Transformer encoder is used, the word order information of the segmented words will be lost, and the relative and absolute position information of each segmented word in the sentence cannot be obtained. The position of the word segmentation in the sentence or chapter is added to the subsequent feature extraction through the above encoding position.

In some embodiments, the fused word vector of the multiple word segments obtained by the first clause combination may be input into the first semantic encoder and feature extraction is performed to obtain the word-level feature vector of the first clause combination, as shown in FIG. 4B . Show. The first semantic vector encoder includes N attention network layers, a first neural network layer and a first single-head attention layer; each of the N attention network layers includes a first multi-head attention layer layer and the first additive normalization layer; N is a positive integer. The process of performing feature extraction on each clause combination by the first semantic vector encoder to obtain the word-level feature vector of each clause combination is shown in Figure 5, including the following steps:

501. Multiple attention modules included in the first multi-head attention layer in the i-th attention network layer respectively perform an attention operation on the fused word vectors of multiple word segmentations included in the first sentence combination to obtain multiple The output of the attention module.

In some embodiments, the first semantic vector encoder includes N layers of first multi-head attention layers, each layer performs a multi-head attention mechanism, and each layer of the first multi-head attention layer includes a plurality of attention modules. The fused word vectors of the multiple segmented words included in the first clause combination are respectively input into multiple attention modules through the multi-head attention mechanism, and the outputs of the multiple attention modules are obtained. For example, the i-th first multi-head attention layer includes h attention modules. The fused word vector of multiple word segmentations is used as the input of the first multi-head attention layer of the i-th layer, and is input into h attention modules. The h attention modules perform attention operations on the fusion word vectors of the input multiple word segmentations, respectively, and obtain the outputs of the h attention modules. Each attention module can use the following formulas (8) to (10) to perform attention operations, and the output result of the i-th attention module can be represented by formula (10).

Q, K, V = C _p ; (8)

Q' _i =QW _i ^Q , K' _i =KW _i ^K , V' _i =VW _i ^V ; (9)

Head _i = Attention(Q′ _i , K′ _i , V′ _i ); (10)

Among them, C _p represents the fusion word vector, Wi ^Q represents the query weight matrix of the _i -th attention module in the first multi-head attention layer of the i-th layer, and Q′ _i represents the first multi-head attention layer of the i-th layer. The query matrix for the ith attention module. W _i ^K represents the key weight matrix of the i-th attention module in the first multi-head attention layer of the i-th layer, and K′ _i represents the key matrix of the i-th attention module in the first multi-head attention layer of the i-th layer . W _i ^V represents the value weight matrix of the i-th attention module in the first multi-head attention layer of the i-th layer, and V′ _i represents the value matrix of the i-th attention module in the first multi-head attention layer of the i-th layer . Head _i represents the output matrix of the ith attention module, and Attention represents the attention operation.

502, the first addition normalization layer in the i-th attention network layer splices the output results of multiple attention modules in each layer to obtain a splicing result; according to the output results of the i-1th attention network layer Perform linear transformation on the splicing result to obtain the first output result of the first multi-head attention layer of the ith attention network layer; the first output result of the first multi-head attention layer of the ith attention network layer Perform normalization processing to obtain the second output result.

In some embodiments, the data form of the output result of the attention module is a matrix, the data form of the splicing result is also a matrix, and the number of dimensions of the splicing result is equal to the sum of the number of dimensions of the output results of each attention module. The splicing method can be horizontal splicing, and the splicing process can be realized by calling the concat (splicing) function. It should be understood that the manner of lateral splicing is only exemplary. Optionally, use other splicing methods to splicing the output results of each attention module, for example, adopting a vertical splicing method to splicing the output results of each attention module to obtain a splicing result, then the number of lines of the splicing result. It is equal to the sum of the number of lines of the output results of each attention module, and the embodiment of the present application does not specifically limit how to perform splicing.

In some embodiments, after the splicing result is obtained, the splicing result may be linearly transformed to obtain the first output result. The method of linear transformation may be multiplication with a weight matrix, the splicing result is multiplied with the weight matrix, and the product is used as the first output result. Optionally, the linear transformation can also use other methods other than multiplying the weight matrix, for example, multiplying the splicing result by a certain constant to perform linear transformation on the splicing result, or multiplying the splicing result with a certain constant. Add, so as to perform linear transformation on the splicing result, and the embodiment of the present application does not specifically limit the method used for the linear transformation.

As an example, in the embodiment of the present application, concact splicing may be used when splicing the output results of the multiple attention modules in the first multi-head attention layer of the ith attention network layer to obtain the splicing result. Then, when performing linear transformation on the splicing result, the first output result may be obtained by multiplying the splicing result by a weight matrix. Among them, the weight matrix is the output result of the i-1th attention network layer. The first output result of the first multi-head attention layer of the i-th attention network layer satisfies the conditions shown in the following formula (11):

表示第i层注意力网络层中h个注意力模块的输出。Among them, W _i ^O represents the weight matrix of the ith attention network layer (that is, the second output result of the ith attention network layer), concat represents the splicing function, and MHA _i (Q, K, V) represents the th The first output result of the attention network layer of layer i, h represents the number of attention modules in the first multi-head attention layer in the attention network layer of layer i, and h is a positive integer greater than 1;

represents the output of h attention modules in the i-th attention network layer.

By using the multi-head attention mechanism, the long-distance features between word segmentation and word segmentation in the text can be captured, rich contextual semantic representation information can be extracted, and the ability to extract features can be enhanced. The vector splicing of the output results of multiple attention modules can be understood as the introduction of original information when calculating the first output result, which can make up for the problem of missing information. In addition, the vector splicing of the output results of multiple attention modules is equivalent to introducing a network path, so that part of the network can be directly propagated to the original information without going through the complex network during backpropagation, preventing gradient explosion. Or the gradient disappears.

In some embodiments, after the first output result is obtained, the first output result is normalized to obtain the second output result. The normalized mean and variance satisfy the conditions shown in the following formulas (12) (13):

代表第i个注意力网络层中第g个注意力模块的输出，μⁱ表示第i层的注意力模块输出的均值，σⁱ表示第i层的注意力模块输出的方差。In the above formula, h represents the number of attention modules in the ith attention network layer,

represents the output of the g-th attention module in the i-th attention network layer, μ ⁱ represents the mean of the output of the attention module of the i-th layer, and σ ⁱ represents the variance of the output of the attention module of the i-th layer.

Each layer normalizes the first output result of the first multi-head attention layer of the i-th attention network layer by the same mean and variance, and obtains the second output result of the i-th layer of attention network layer. For example, the second output result of the i-th attention network layer can be represented by M′=(m′ ₁ ,m′ ₂ ,m′ ₃ ,...,m′ _n ), then m′ _j satisfies the following formula (14) Conditions shown:

Among them, m' _j represents the j-th vector in the second output result of the i-th attention network layer, m' _j ∈ R ^m , m represents the dimension of m' _j , and m _j represents the i-th layer of attention network layer The jth vector in the first output of the first multi-head attention layer.

After the above normalization, the data distribution can be relatively consistent. In the process of propagation, the network often deviates, which makes back propagation difficult. A normalization operation is performed on each layer, and after normalization, the second output result of each layer is in line with the normal distribution. The feature of normalization is that it does not depend on the number of input sequences and the length of the input sequence, and it has an improved effect on the neural network.

以此类推，可以将第i层注意力网络层的第二输出结果M′_i用于第i+1个注意力网络层的线性变换中的权重矩阵

如图6所示。权重矩阵满足如下公式所示的条件：In some embodiments, after obtaining the second output result of the i-th attention network layer, the second output result of the i-th attention network layer can be used for linear transformation of the i+1-th attention network layer. For example, after the fusion word vector of multiple word segments included in the first sentence combination is input to the first attention network layer to obtain the second output result of the first attention network layer, the obtained second output of the first layer can be The result is the weight matrix of the linear transformation in the second attention network layer. For example, the second output result of the first layer can be expressed as M′ ₁ , and M′ ₁ can be used as the weight matrix of the linear transformation in the second attention network layer

By analogy, the second output result M′ _i of the i-th attention network layer can be used for the weight matrix in the linear transformation of the i+1-th attention network layer

As shown in Figure 6. The weight matrix satisfies the conditions shown in the following formula:

表示第2个注意力网络层的权重矩阵，M′₁表示第1个注意力网络层的第二输出结果，MHA₂(Q,K,V)表示第2层注意力网络层的第一输出结果，W_i ^O表示第i个注意力网络层的权重矩阵，M′_i-1表示第i-1个注意力网络层的第二输出结果，MHA_i(Q,K,V)表示第i层注意力网络层的第一输出结果，

表示第i+1个注意力网络层的权重矩阵，M′_i示第i个注意力网络层的第二输出结果，MHA_i+1(Q,K,V)表示第i+1层注意力网络层的第一输出结果。in,

Represents the weight matrix of the second attention network layer, M' ₁ represents the second output result of the first attention network layer, MHA ₂ (Q, K, V) represents the first output of the second attention network layer As a result, W _i ^O represents the weight matrix of the i-th attention network layer, M′ _i-1 represents the second output result of the i-1-th attention network layer, and MHA _i (Q, K, V) represents the i-th The first output result of the layer attention network layer,

Represents the weight matrix of the i+1th attention network layer, M′ _i represents the second output result of the ith attention network layer, and MHA _i+1 (Q, K, V) represents the i+1th layer of attention The first output result of the network layer.

503. Perform feature extraction on the second output result of the Nth attention network layer through the first neural network layer, to obtain a feature matrix of the first segmented word of each clause combination.

In some embodiments, after obtaining the second output result of the Nth attention network layer, the second output result of the Nth attention network layer may be linearly transformed or nonlinearly transformed through the first neural network to obtain The feature matrix of the first participle of each clause combination. The linear transformation may include operations of multiplying a matrix and adding an offset, and the nonlinear transformation may be implemented by a nonlinear function. For example, a nonlinear transformation can be a maximization operation. For example, it can be implemented using the max function. The max function is only an exemplary implementation of nonlinear transformation, and other methods can also be used to perform nonlinear transformation, such as performing operations through an activation function, so as to realize nonlinear transformation. Make specific restrictions. As an example, the first neural network in the embodiment of the present application may be implemented by a feedforward neural network. Specifically, a two-layer fully connected network may be used to perform feature extraction on the second output result of the Nth attention network layer, The feature matrix of the first participle of each clause combination satisfies the conditions shown in the following formula (15):

M _d =Relu((M'W ¹ +b ₁ )W ² +b ₂ ); (15)

Among them, M′ represents the second output result of the Nth attention network layer, M _d represents the feature matrix of the first word segmentation, W ¹ , W ² represent the parameter matrix, W ¹ , W ² ∈ R ^m×m , b ₁ , b ₂ represents the bias term, and Relu represents the activation function.

504. Extract feature information in the feature matrix of the first word segment through the first single-head attention layer, and obtain a word-level feature vector of the first clause combination.

In some embodiments, a trainable attention vector can be used to replace the Q query vector in the existing single-head attention layer to extract the feature information in the feature matrix of the first segmented word, so as to obtain the word level of the first segmented sentence combination. Feature vector. The word-level feature vector satisfies the conditions shown in the following formulas (16)-(18):

a _i = u _w · m _di ; (16)

Among them, a _i is the feature matrix weighted by the attention mechanism, a′ _i is the feature matrix normalized by softmax, u _w is the initialized attention vector, m _di is the first word in the feature matrix of the first segment i column vectors, u _w ∈ R ^m , s _j represents the word-level feature vector, s _j ∈ R ^m , and R ^m is an m×m matrix.

In some embodiments, feature splicing may be performed on word-level feature vectors of multiple clause combinations to obtain a sentence vector representation of each clause combination. For example, when the first clause combination includes n word-level feature vectors, the sentence vector of the first clause can be expressed as S=(s ₁ , s ₂ , s ₃ , . . . , s _n ).

In some embodiments, feature extraction may be performed by using the sentence vector of each clause combination by the second semantic vector encoder, so as to obtain a clause-level feature vector of each clause combination. In some scenarios, before the feature extraction is performed on the sentence vector, position vectors corresponding to the plurality of first sentence vectors included in the filtered first chapter may be encoded. The position vector corresponding to the first sentence vector is used to represent the position of the clause corresponding to the first sentence vector in the text corresponding to the first chapter.

In some embodiments, after the position vector corresponding to the first sentence vector is obtained, the position vector of the first sentence vector is fused with the first sentence vector to obtain a fused sentence vector. For the specific method, reference may be made to the encoding method of the above-mentioned fusion word vector, which will not be repeated here. Further, the fused sentence vectors corresponding to multiple sentences can be input into the second semantic vector encoder and feature extraction is performed to obtain sentence-level feature vectors of the first sentence combination, as shown in FIG. 4B .

In some embodiments, the second semantic vector encoder includes N attention network layers, a second neural network layer, and a second single-head attention layer; each of the N attention network layers includes a Two multi-head attention layers and a second additive normalization layer. The second semantic vector encoder is used to perform feature extraction on the first clause combination according to the fused sentence vector of the multiple clauses included in the first clause combination, so as to obtain the clause-level feature vector of the first clause combination, including The following steps:

601. Multiple attention modules included in the second multi-head attention layer in the i-th network attention layer respectively perform attention calculation on the fused sentence vectors of the multiple sentences included in the first sentence combination, so as to obtain the result. output of multiple attention modules.

602, the second addition normalization layer in the i-th network attention layer splices the output results of the multiple attention modules in each layer to obtain a splicing result; according to the output results of the i-1 th attention network layer Perform linear transformation on the splicing result to obtain the third output result of the second multi-head attention layer according to the i-1th attention network layer; i is a positive integer less than or equal to N; The third output result of the second multi-head attention layer of the layer is normalized to obtain the fourth output result.

In some embodiments, the fourth output result can be used for linear transformation of the i+1 th attention network layer.

603. Perform feature extraction on the fourth output result of the Nth attention network layer through the second neural network layer, to obtain a feature matrix of the first clause of each clause combination.

604. Extract the feature information in the feature matrix of the first clause through the second single-head attention layer, and obtain the clause-level feature vector of the first clause combination.

The specific methods of the above steps 601-604 are the same as the methods of steps 501-504, and are not repeated here.

In some embodiments, after obtaining clause-level feature vectors of multiple clause combinations, feature splicing may be performed on the clause-level feature vectors of multiple clause combinations to obtain a first feature vector corresponding to each chapter.

In some embodiments, after obtaining the first feature vector corresponding to each chapter, the predicted category of each chapter is determined according to the first feature vector. Therefore, the information of multiple chapters can be stripped, and the prediction categories can be determined as multiple chapters corresponding to the categories of laws, charges, and sentences respectively.

After the prediction category of each chapter is determined, the sentence prediction, the law prediction and the crime prediction may be performed by combining the second feature vector, the first feature vector of each chapter, and the prediction category. Exemplarily, as shown in FIG. 7 , the sentencing prediction device can obtain the law prediction vector through nonlinear transformation according to the second feature vector and the plurality of first feature vectors corresponding to the plurality of chapters whose prediction category is the law category. Bar prediction vector for normal bar prediction. Perform nonlinear transformation according to the second feature vector, multiple first feature vectors corresponding to multiple chapters whose predicted category is the crime category, and the crime prediction vector to obtain the crime prediction vector, and perform the crime prediction according to the crime prediction vector. Perform nonlinear transformation according to the second eigenvector, multiple first eigenvectors corresponding to multiple chapters whose prediction category is the sentence category, the law prediction vector, and the crime prediction vector to obtain the sentence prediction vector, and predict the sentence according to the sentence prediction vector.

In an example, execution of sentence prediction, law prediction and crime prediction may be implemented by a classifier, and the classifier may include a law prediction network, a crime prediction network, and a sentence prediction network. See Figure 8. The law prediction network, the crime prediction network and the sentence prediction network can use the forward propagation network. Referring to Fig. 3 and Fig. 4B, the predicted category corresponding to each chapter is obtained through the network that encodes the crime fact description text corresponding to Fig. 4B, and after obtaining the second feature vector through the network corresponding to Fig. 3, the second feature vector and Multiple first feature vectors corresponding to multiple chapters whose predicted category is the legal category are spliced (see formula (19)) to obtain the first splicing vector, and input the first splicing vector into the classifier (see formula (20) and (21)), to make predictions on legal provisions. Under normal circumstances, the statutes promulgated by the law include multiple clauses, and different clauses can be set as a clause category. The calculation process of the law prediction satisfies the conditions shown in the following formulas (19)-(21):

为第一拼接向量，T_m为第二特征向量，T₁为预测类别为法条类别的多个篇章对应的第一特征向量，

为法条预测向量，T_l法条预测中每个条款类别的概率分布，T_l∈R^x，x为法条预测的第一特征向量的数量，

W_l ¹为权重，

为偏置项，Relu为激活函数。in,

is the first splicing vector, T _m is the second feature vector, T ₁ is the first feature vector corresponding to multiple chapters whose prediction category is the legal category,

is the rule prediction vector, T _l is the probability distribution of each item category in the rule prediction, T _l ∈ R ^x , x is the number of the first eigenvectors of the rule prediction,

W _l ¹ is the weight,

is the bias term, and Relu is the activation function.

In some embodiments, the input vector of the crime prediction includes T _m , T ₂ and the rule prediction vector T _l ¹ . Splicing T _m , T ₂ , and T _l ¹ to obtain a second splicing vector (see formula (22)), and inputting the second splicing vector into the classifier (see formula (23) and formula (24)) to predict the crime . Under normal circumstances, charges may include multiple types, and different charges can be set as one charge category. The calculation process of the crime prediction satisfies the conditions shown in the following formulas (22)-(24):

表示第二拼接向量，T_m表示第二特征向量，T_l ¹表示法条预测向量，T₂表示预测类别为罪名类别的多个篇章对应的第一特征向量，

表示罪名预测向量，T_ch表示罪名预测中每个罪名类别的概率分布，T_ch∈R^y，y为罪名预测的第一特征向量的数量，

为权重，

为偏置项，Relu为激活函数。in,

represents the second splicing vector, T _m represents the second feature vector, T _l ¹ represents the law prediction vector, T ₂ represents the first feature vector corresponding to multiple chapters whose prediction category is the crime category,

represents the crime prediction vector, T _ch represents the probability distribution of each crime category in the crime prediction, T _ch ∈ R ^y , y is the number of the first feature vector of the crime prediction,

is the weight,

is the bias term, and Relu is the activation function.

将T_m,T₂,T_l ¹,

进行拼接获得第三拼接向量(参见公式(25))，将第三拼接向量输入分类器中(参见公式(26)和公式(27))，进行刑期预测。一般情况下，刑期可能包括多种期限，比如5年、10年、不判刑、死刑以及无期。不同的刑期可被设定为一个期限类别。刑期预测的计算过程满足如下公式(25)-(27)所示的条件：In some embodiments, the input vector of sentence prediction includes T _m , T ₃ , T _l ¹ and the crime prediction vector

Set T _m , T ₂ , T _l ¹ ,

Perform splicing to obtain a third splicing vector (see formula (25)), input the third splicing vector into the classifier (see formula (26) and formula (27)), and perform sentence prediction. Under normal circumstances, the sentence may include a variety of terms, such as 5 years, 10 years, no sentence, the death penalty and life. Different sentences can be set as a term category. The calculation process of sentence prediction satisfies the conditions shown in the following formulas (25)-(27):

表示第二拼接向量，T_m表示第二特征向量，T_l ¹表示法条预测向量，

为罪名预测向量，T₃表示预测类别为刑期类别的多个篇章对应的第一特征向量，

表示刑期预测向量，T_p表示刑期预测中每个期限类别的概率分布，T_p∈R^z，z为刑期预测的第一特征向量的数量，

为权重，

为偏置项，Relu为激活函数。in,

represents the second splicing vector, T _m represents the second feature vector, T _l ¹ represents the normal prediction vector,

is the crime prediction vector, _T3 represents the first feature vector corresponding to multiple chapters whose prediction category is the sentence category,

represents the sentence prediction vector, T _p represents the probability distribution of each term category in the sentence prediction, T _p ∈ R ^z , z is the number of the first feature vector of the sentence prediction,

is the weight,

is the bias term, and Relu is the activation function.

In some embodiments, the sentencing prediction model may be trained with multiple samples in the training set. Samples include adjudication documents and case-related information. Among them, the judgment documents include document information, defendant information, description of criminal facts, and court judgment information. Among them, the document information is the abstract and title of the criminal judgment document, and the court judgment information includes the fact-finding part information and label information. Fact determination includes conclusion information, amount information, plot information, consequence information, and confession attitude information. Labels are extracted into three categories: various clause categories in relevant laws and regulations, multiple crime categories included in the crime (for example, including innocence), and sentence (for example, including no sentence, death penalty, and life sentence). When training the sentencing prediction model, multiple samples can be input into the sentencing prediction model through multiple iterations, one sample can be input at a time, and the predicted result of the law for the sample output by the sentencing prediction model and the sample label The prediction results of the legal provisions in the sentence are compared to adjust the various network parameters in the sentencing prediction model. Each network parameter in the sentencing prediction model is adjusted according to the comparison between the sentence prediction result of the sample output by the sentencing prediction model and the sentence label in the sample label. Each network parameter in the sentencing prediction model is adjusted according to the comparison between the crime prediction result of the sample output by the sentencing prediction model and the crime label in the sample label.

In a possible example, when the network parameters are adjusted, the network parameters may be adjusted by determining the comparison result (ie the loss value) obtained by the comparison through the loss function. The loss function can be, for example, a cross-entropy loss function.

The loss of each prediction task can be obtained using the cross-entropy loss function, and the expression of the cross-entropy loss function satisfies the conditions shown in the following formula (28):

表示预测结果，即为属于当前类别的概率，

为预测第i个样本为类别l的损失值。取值为1或2或3用来标识罪名预测的损失值、法条预测的损失值或者刑期预测的损失值。比如1用于标识罪名预测，2用于标识法条预测，3用于标识刑期预测。Among them, y indicates whether the data belongs to the current category. For example, the value of y can be 0 or 1, and the value of l indicates the category, and the value is 1 or 2 or 3.

represents the prediction result, that is, the probability of belonging to the current category,

is the loss value for predicting the i-th sample as category l. A value of 1 or 2 or 3 is used to identify the predicted loss value of the crime, the predicted loss value of the law or the predicted loss value of the sentence. For example, 1 is used to identify the crime prediction, 2 is used to identify the legal article prediction, and 3 is used to identify the sentence prediction.

In some embodiments, the loss value determined by the loss function can be predicted for the crime, the loss value determined by the loss function for the prediction of the law, and the loss value determined by the loss function for the sentence prediction can be accumulated to obtain the total loss value, and then based on the total loss value. loss value to adjust the network parameters of the sentencing prediction model. In some scenarios, network parameters can be adjusted through an optimization algorithm. For example, the network parameters of the sentencing prediction model can be adjusted through the Adam optimization algorithm.

In some possible implementations, the different networks in the sentencing prediction model are trained separately. For example, the network in Figure 4B for encoding crime fact description text is separately trained. Each of the multiple samples in the training set of the network encoding the description text of the crime facts may include the judgment document and a label corresponding to each sentence or paragraph in the judgment document, and the label is used to indicate that the label to which the sentence or paragraph belongs is a crime category, sentence category, and statute category. When training the network that encodes the description text of the crime facts, multiple samples can be input into the network that encodes the description text of the crime facts through multiple iterations. One sample can be input at a time, and according to the output of the network that encodes the description text of the crime facts The prediction result of the sample is compared with the categories in the sample label (criminal category, sentence category and legal article category) to obtain the comparison result, and each network parameter in the network encoding the description text of the crime facts is adjusted by comparing the results.

In a possible example, when the network parameters are adjusted, the network parameters may be adjusted by determining the comparison result obtained by the comparison through the loss function. The loss function can be, for example, a cross-entropy loss function.

The loss of each prediction task can be obtained using the cross-entropy loss function, and the expression of the cross-entropy loss function satisfies the conditions shown in the following formula (29):

表示预测结果，即为属于当前类别的概率，

为预测第i个样本为类别l的损失。取值为1或2或3用来标识罪名类别、法条类别或者刑期类别。比如1用于标识罪名类别，2用于标识法条类别，3用于标识刑期类别。Among them, y indicates whether the data belongs to the current category (criminal category, law category or sentence category), for example, the value of y can be 0 or 1, l indicates the category, and the value is 1 or 2 or 3,

represents the prediction result, that is, the probability of belonging to the current category,

is the loss for predicting that the ith sample is class l. A value of 1 or 2 or 3 is used to identify the crime category, law category or sentence category. For example, 1 is used to identify the crime category, 2 is used to identify the legal article category, and 3 is used to identify the sentence category.

In some embodiments, during the training process, the data of the probability distribution of predicted categories may be represented by a set of numbers consisting of 1s and 0s, and the data refers to the combination of multiple chapters and the predicted classification results. For example, each chapter can be assigned a serial number. When the probability distribution data is [35273, label1], it means the 35273rd chapter, and the predicted category label is the first category. If the prediction result is consistent with the result of manual annotation, it is marked as 1, otherwise, it is 0.

Based on the same technical concept, an embodiment of the present application provides a sentencing prediction apparatus 800, as shown in FIG. 9 . The device 800 can execute each step in the above-mentioned sentencing prediction method, which will not be described in detail here in order to avoid repetition. The apparatus 800 includes an obtaining unit 801 and a processing unit 802, including an obtaining unit and a processing unit;

The obtaining unit 801 is configured to obtain case-related information and a description text of the criminal facts, the case-related information includes at least one of witness evidence, physical evidence, defendant information, testimony, suspect population testimony, and transcripts; the criminal facts description The text includes a plurality of chapters, each of the plurality of chapters includes a plurality of clauses, and each of the plurality of sentences includes a plurality of participles;

The processing unit 802 is configured to perform vectorization processing on the word segments included in the plurality of chapters to obtain a first word vector corresponding to each word segment, and perform vectorization processing on the word segments included in the case-related information to obtain each word segment. The second word vector corresponding to the word segmentation; perform feature extraction on the first word vector included in the multiple chapters to obtain the first feature vector of each chapter in the multiple chapters, and determine the first feature vector according to the first feature vector of each chapter The predicted category of each chapter, the predicted category is the category of the law or the category of the crime or the category of the sentence;

The processing unit 802 is further configured to perform feature extraction on the second word vector included in the case-related information to obtain a second feature vector of the case-related information; according to the multiple first feature vectors corresponding to the multiple chapters The prediction category of , and the second feature vector are used to predict legal articles, charges and sentences.

In some embodiments, when the processing unit 802 performs feature extraction on the first word vectors included in the multiple chapters to obtain the first feature vector of each chapter in the multiple chapters, the processing unit 802 is specifically configured to: based on the first chapter The included first word vector filters the word segmentation in the first chapter to obtain the filtered first chapter, and the first word vectors in the multiple clauses included in the filtered first chapter are all related to sentencing prediction, The first chapter is any one of the multiple chapters; the multiple clauses included in the filtered first chapter are combined to obtain multiple clause combinations, each of the multiple clause combinations. A clause combination consists of at least two clauses;

The first semantic vector encoder is used to extract features for each clause combination to obtain word-level feature vectors of each clause combination; perform feature splicing on word-level feature vectors of multiple clause combinations to obtain each segment Sentence vector representation of sentence combination;

The second semantic vector encoder performs feature extraction on the sentence vector of each clause combination to obtain the clause-level feature vector of each clause combination; and performs feature splicing on the clause-level feature vectors of multiple clause combinations to obtain the first feature vector.

In some embodiments, the processing unit 802 is further configured to: encode a position vector corresponding to a plurality of first word vectors included in the filtered first chapter, where the position vector corresponding to the first word vector is used to represent the The position of the word segment corresponding to the first word vector in the text corresponding to the first chapter; the first word vector of the multiple word segments included in the filtered first chapter and the corresponding position vector are fused to obtain the first word vector in the first chapter. The fusion word vector of multiple word segmentations;

The processing unit 802, when performing feature extraction on each clause combination by the first semantic vector encoder, to obtain a word-level feature vector of each clause combination, is specifically used for: The fusion word vector of multiple word segments uses the first semantic vector encoder to perform feature extraction on the first clause combination, so as to obtain the word-level feature vector of the first clause combination, and the first clause combination is the multi-sentence combination. any one of these clause combinations.

In other embodiments, the processing unit 802 is further configured to: encode a position vector corresponding to a plurality of first sentence vectors included in the filtered first chapter, where the position vector corresponding to the first sentence vector is used to represent the Describe the position of the clause corresponding to the first sentence vector in the text corresponding to the first chapter; fuse the plurality of first sentence vectors included in the filtered first chapter with the corresponding position vector to obtain the first sentence in the first chapter. The fused sentence vector of multiple clauses;

The processing unit 802, when performing feature extraction on the sentence vector of each sentence combination by the second semantic vector encoder, to obtain the sentence-level feature vector of each sentence combination, is specifically used for: according to the first score. The fused sentence vector of the multiple clauses included in the sentence combination adopts the second semantic vector encoder to perform feature extraction on the first clause combination, so as to obtain the clause-level feature vector of the first clause combination, and the first clause The sentence combination is any one of the plurality of clause combinations.

In some embodiments, the first semantic vector encoder includes N attention network layers, a first neural network layer, and a first single-head attention layer; each attention network layer in the N attention network layers Including the first multi-head attention layer and the first addition normalization layer; N is a positive integer;

The processing unit 802 uses the first semantic vector encoder to perform feature extraction on the first clause combination according to the fused word vector of the plurality of word segments included in the first clause combination, so as to obtain the words of the first clause combination. level feature vector, it is specifically used for: the fusion of the multiple word segmentations included in the first sentence combination by the multiple attention modules included in the first multi-head attention layer in the i-th attention network layer. The word vector performs an attention operation to obtain the output of the multiple attention modules; the first addition normalization layer in the i-th attention network layer performs the output results of the multiple attention modules. Splicing to obtain a splicing result; performing linear transformation on the splicing result according to the output result of the i-1th attention network layer, to obtain the first output result of the first multi-head attention layer of the ith attention network layer; i is a positive integer less than or equal to N; the first output result of the first multi-head attention layer of the ith attention network layer is normalized to obtain a second output result, the second output result Linear transformation for the i+1 th attention network layer; feature extraction is performed on the second output result of the N th attention network layer through the first neural network layer to obtain the th The feature matrix of a segmented word; the feature information in the feature matrix of the first segmented word is extracted through the first single-head attention layer, and the word-level feature vector of the first clause combination is obtained.

In other embodiments, the second semantic vector encoder includes N attention network layers, a second neural network layer and a second single-head attention layer; each attention network in the N attention network layers The layer includes a second multi-head attention layer and a second additive normalization layer; N is a positive integer;

The processing unit 802 uses the second semantic vector encoder to perform feature extraction on the first clause combination according to the fusion sentence vector of the multiple clauses included in the first clause combination, so as to obtain the first clause combination. When a sentence-level feature vector is used, it is specifically used for: the plurality of attention modules included in the second multi-head attention layer in the i-th attention network layer are respectively used for the plurality of divisions included in the first sentence combination. The fusion sentence vector of the sentence performs attention calculation to obtain the output of the plurality of attention modules; the second addition normalization layer in the i-th attention network layer The output results are spliced to obtain the splicing result; the splicing result is linearly transformed according to the output result of the i-1th attention network layer, and the third multi-head attention layer of the second multi-head attention layer of the i-th attention network layer is obtained. Output result; i is a positive integer less than or equal to N; the third output result of the second multi-head attention layer of the i-th attention network layer is normalized to obtain the fourth output result, and the fourth output result is obtained. The four output results are used for the linear transformation of the i+1th attention network layer; the feature extraction is performed on the fourth output result of the Nth attention network layer through the second neural network layer, and each sentence is obtained. The feature matrix of the combined first clause; the feature information in the feature matrix of the first clause is extracted through the second single-head attention layer, and the clause-level feature vector of the first clause combination is obtained.

In some embodiments, the processing unit 802, when performing law prediction, crime prediction and sentence prediction according to the prediction categories of the plurality of first feature vectors corresponding to the plurality of chapters and the second feature vector, specifically uses In: performing nonlinear transformation on the second eigenvector and the first eigenvectors corresponding to the plurality of chapters and the first eigenvector whose prediction category is the law category to obtain a law prediction vector, and according to the law The prediction vector is used to predict the law; the second feature vector, the plurality of first feature vectors corresponding to the plurality of chapters, the first feature vector whose prediction category is the crime category, and the law prediction vector are nonlinearly transformed Obtain a crime prediction vector, and perform crime prediction according to the crime prediction vector; for the second feature vector and the first feature vectors corresponding to the multiple chapters, the predicted category is the first feature vector of the sentence category, and the The law prediction vector and the crime prediction vector are nonlinearly transformed to obtain a sentence prediction vector, and the sentence prediction is performed according to the sentence prediction vector.

In other embodiments, the case-related information includes first data and second data; wherein the first data includes at least one of testimony, a suspect's oral testimony, and a transcript, and the second data includes witnesses, At least one of physical evidence and defendant information; the processing unit 802, when performing vectorization processing on the word segmentation included in the case-related information to obtain the second word vector corresponding to each word segmentation, is specifically used for: Perform vectorization processing on the word segments included in the first data to obtain a second word vector corresponding to each segment in the first data; determine the category to which each word segment included in the second data belongs, and determine the first word vector from the data vector table The category vector corresponding to the category to which each participle included in the second data belongs; the data vector table includes category vectors corresponding to multiple categories; the category vector corresponding to the category to which each participle included in the second data belongs is determined as the The second word vector corresponding to each participle.

In some embodiments, when the processing unit 802 performs filtering processing on the word segmentation in the first chapter based on the first word vector included in the first chapter to obtain the filtered first chapter, the processing unit 802 is specifically configured to: The network performs filtering processing on the plurality of first word vectors included in the first chapter to obtain the filtered first chapter.

Based on the same technical concept, an embodiment of the present application provides a sentencing prediction apparatus 1000, as shown in FIG. 10 . The apparatus 1000 can execute each step in the above-mentioned sentencing prediction method, which will not be described in detail here in order to avoid repetition. The apparatus 1000 includes a memory 1001 and a processor 1002 .

The memory 1001 is used to store program instructions;

The processor 1002 is configured to call the program instructions stored in the memory, and execute the above sentencing prediction method according to the obtained program.

An embodiment of the present application provides a computer-readable storage medium, where computer instructions are stored in the computer-readable storage medium, and when the computer instructions are executed on a computer, the computer is made to execute the first aspect and the components included in the first aspect. Any possible implementation of the method described.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims (10)

1. A crime prediction method, comprising:

acquiring case related information and a criminal fact description text, wherein the case related information comprises at least one of testimony, material evidence, informed person information, testimony, mouth supply of a suspect and a record; the crime fact description text comprises a plurality of chapters, each chapter of the plurality of chapters comprises a plurality of clauses, and each clause of the plurality of sentences comprises a plurality of participles;

vectorizing the participles included in the plurality of chapters to obtain a first word vector corresponding to each participle, and vectorizing the participles included in the case related information to obtain a second word vector corresponding to each participle;

performing feature extraction on the first word vectors included in the plurality of chapters to obtain a first feature vector of each chapter in the plurality of chapters, and determining a prediction category of each chapter according to the first feature vector of each chapter, wherein the prediction category is a law article category or a criminal name category or a criminal period category;

performing feature extraction on a second word vector included in the case related information to obtain a second feature vector of the case related information;

and performing normal prediction, criminal name prediction and criminal phase prediction according to the prediction categories of the first feature vectors corresponding to the chapters and the second feature vectors.

2. The method of claim 1, wherein feature extracting the first word vector included in the plurality of chapters to obtain the first feature vector of each chapter of the plurality of chapters comprises:

filtering the participles in a first chapter based on a first word vector included in the first chapter to obtain a filtered first chapter, wherein the first word vector in a plurality of clauses included in the filtered first chapter is related to sentry prediction, and the first chapter is any one of the plurality of chapters;

combining the multiple clauses included in the filtered first discourse to obtain multiple clause combinations, wherein each clause combination in the multiple clause combinations comprises at least two clauses;

extracting the characteristics of each clause combination through a first semantic vector encoder to obtain a word-level characteristic vector of each clause combination;

performing feature splicing on the word-level feature vectors of the multiple clause combinations to obtain statement vector representation of each clause combination;

performing feature extraction on the sentence vector of each sentence combination through a second semantic vector encoder to obtain a sentence-level feature vector of each sentence combination;

and performing feature splicing on the sentence-level feature vectors of the plurality of sentence combinations to obtain the first feature vector.

3. The method of claim 2, wherein the method further comprises:

encoding position vectors corresponding to a plurality of first word vectors included in the filtered first chapters, wherein the position vectors corresponding to the first word vectors are used for representing positions of participles corresponding to the first word vectors in texts corresponding to the first chapters;

fusing the first word vectors of the multiple participles included in the filtered first discourse with the corresponding position vectors to obtain fused word vectors of the multiple participles in the first discourse;

the extracting the feature of each clause combination through the first semantic vector encoder to obtain the word-level feature vector of each clause combination comprises the following steps:

and performing feature extraction on a first sentence combination by adopting a first semantic vector encoder according to a fused word vector of a plurality of clauses included in the first sentence combination to obtain a word-level feature vector of the first sentence combination, wherein the first sentence combination is any one of the plurality of sentence combinations.

4. The method of claim 2, wherein the method further comprises:

encoding position vectors corresponding to a plurality of first statement vectors included in the filtered first chapters, wherein the position vectors corresponding to the first statement vectors are used for representing positions of clauses corresponding to the first statement vectors in texts corresponding to the first chapters;

fusing the plurality of first sentence vectors included in the filtered first chapters with corresponding position vectors to obtain fused sentence vectors of a plurality of clauses in the first chapters;

the extracting the feature of the sentence vector of each sentence combination by the second semantic vector encoder to obtain the sentence-level feature vector of each sentence combination includes:

and performing feature extraction on the first sentence combination by adopting a second semantic vector encoder according to a fusion sentence vector of a plurality of sentences included in the first sentence combination to obtain a sentence-level feature vector of the first sentence combination, wherein the first sentence combination is any one of the plurality of sentence combinations.

5. The method of any of claims 1-4, wherein said performing forensic prediction, criminal name prediction and criminal phase prediction based on the prediction categories of the first plurality of feature vectors corresponding to the plurality of chapters and the second feature vector comprises:

performing nonlinear transformation on the second feature vector and a first feature vector of which the prediction category is a normal category in a plurality of first feature vectors corresponding to the plurality of chapters to obtain a normal prediction vector, and performing normal prediction according to the normal prediction vector;

carrying out nonlinear transformation on the second feature vector, a first feature vector with a prediction category being a guilty category in a plurality of first feature vectors corresponding to the plurality of chapters and the normal prediction vector to obtain a guilty prediction vector, and carrying out guilty prediction according to the guilty prediction vector;

and carrying out nonlinear transformation on the second feature vector, the first feature vector with the prediction category being the criminal category in the plurality of first feature vectors corresponding to the plurality of chapters, the normal prediction vector and the criminal name prediction vector to obtain a criminal prediction vector, and carrying out criminal prediction according to the criminal prediction vector.

6. The method according to any of the claims 1-4, wherein said case related information comprises first data and second data; the first data comprise at least one item of testimony words, suspected person oral supply and notes, and the second data comprise at least one item of testimony, material evidence and information of an advertiser; the vectorizing processing of the participles included in the case related information to obtain a second word vector corresponding to each participle includes:

vectorizing the participles included in the first data to obtain a second word vector corresponding to each participle in the first data;

determining the category to which each participle included in the second data belongs, and determining a category vector corresponding to the category to which each participle included in the second data belongs from a data vector table; the data vector table comprises category vectors corresponding to a plurality of categories; and determining the second word vector corresponding to each participle according to the category vector corresponding to the category to which each participle included in the second data belongs.

7. The method of any of claims 1-4, wherein the filtering the participles in the first chapter based on the first word vector comprised by the first chapter to obtain a filtered first chapter comprises:

filtering, by a convolutional neural network, a plurality of first word vectors included in the first chapters to obtain the filtered first chapters.

8. A crime prediction apparatus, comprising an acquisition unit and a processing unit;

the acquisition unit is used for acquiring case related information and criminal fact description texts, wherein the case related information comprises at least one of testimony, material evidence, information of an advertiser, testimony, oral supply of a suspect and a note; the crime fact description text comprises a plurality of chapters, each chapter of the plurality of chapters comprises a plurality of clauses, and each clause of the plurality of sentences comprises a plurality of participles;

the processing unit is used for vectorizing the participles included in the plurality of chapters to obtain a first word vector corresponding to each participle, and vectorizing the participles included in the case related information to obtain a second word vector corresponding to each participle; performing feature extraction on the first word vectors included in the plurality of chapters to obtain a first feature vector of each chapter in the plurality of chapters, and determining a prediction category of each chapter according to the first feature vector of each chapter, wherein the prediction category is a law article category or a criminal name category or a criminal period category;

the processing unit is further configured to perform feature extraction on a second word vector included in the case related information to obtain a second feature vector of the case related information; and performing normal prediction, criminal name prediction and criminal phase prediction according to the prediction categories of the first feature vectors corresponding to the chapters and the second feature vectors.

9. A crime prediction apparatus comprising a memory and a processor;

the memory to store program instructions;

the processor, for calling program instructions stored in the memory, for executing the method of any one of claims 1-7 according to the obtained program.

10. A computer-readable storage medium having stored thereon computer instructions which, when executed on a computer, cause the computer to perform the method of any one of claims 1-7.

Images