RU2760637C1 - Method and system for retrieving named entities - Google Patents

Abstract

FIELD: computing technology.

SUBSTANCE: disclosed is a method for retrieving named entities from textual information, implemented by at least one computing apparatus, comprising the stages wherein: the textual information is obtained; the text is divided into words; the text is tokenised to obtain a token sequence; a set of vectors is formed by means of a neural network for the obtained token sequence; a vector representation of the token sequence is formed based on the obtained set of vectors; by comparing the indicators of the obtained vector representation of the token sequence with the predefined vector indicators obtained as a result of training of the neural network, named entities are predicted for the vector representation of the token sequence; the named entities obtained at the previous stage are identified by selecting the word mark.

EFFECT: increase in the accuracy of prediction of named entities.

8 cl, 2 dwg

Description

FIELD OF TECHNOLOGY

[0001] The presented technical solution relates generally to the field of computing, and in particular to a method and system for extracting named entities. The technical solution finds application in many areas related to the processing of texts in natural languages and the extraction of information from the text.

LEVEL OF TECHNOLOGY

[0002] Currently, the task of extracting named entities from text data is required in a wide range of areas and applications. In particular, when building dialog systems (chat bots, smart assistants), the extraction of named entities is necessary for correctly understanding the meaning of the text, the user's intentions and forming a response to user requests.

[0003] In industrial areas where document classification is required, named entity extraction can be used to generate new text features to improve the performance of the classification in a real-world environment.

[0004] The task of extracting named entities is a subtask in a number of areas, such as parsing text documents, classifying reviews, clustering text collections, in dialog systems, etc.

[0005] Currently, a number of approaches are used to solve the problem of extracting named entities from Russian texts, each of which has its own advantages and disadvantages:

- Named Entity Recognition (NER) from DeepPavlov, URL: http://docs.deeppavlov.ai/en/master/features/models/ner.html. The solution is based on additional training of the RuBERT language model for the task of extracting named entities. The system has over 100 million learning parameters. Learning with this model requires large amounts of GPU memory (over 3 GB);

- Application of a Hybrid Bi-LSTM-CRF model to the task of Russian Named Entity Recognition. URL: https://arxiv.org/pdf/1709.09686.pdf. The solution is based on the concatenation of a Bi-LSTM char layer, a Token Embedding layer for encoding the input sequence, and a Bi-LSTM-CRF layer for entity prediction. This solution shows worse quality than modern models based on large language models, however, it takes up much less space in memory during training;

- Deep-NER: named entity recognizer based on deep neural networks and transfer learning. The model is based on ELMO (Matthew Ε. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer. Deep contextualized word representations. 2018. URL: https://arxiv.org/abs/1802.05365) or BERT language model. Next comes the BiLSTM-CRF prediction layer. The system has more than 100 million training parameters and occupies more than 3-4 GB of GPU memory in memory during training. Works worse than DeepPavlov's model;

- Tomita parser. URL: https://vandex.ru/dev/tomita/. Tomita parser is designed to extract structured data from natural language text. Facts are extracted using context-free grammars and keyword dictionaries. The parser allows you to write your own grammars and add dictionaries for the desired language. It can be used to retrieve named entities. The main disadvantage is the complexity of writing the rules: for each new entity, a new group of rules is required, which is time-consuming for a specialist. However, this tool does not require training data.

[0006] The main disadvantages of the above neural network models are that they require large amounts of memory for training and prediction and the availability of appropriate equipment. Using a tomita parser requires a highly skilled expert to write grammars.

DISCLOSURE OF THE INVENTION

[0007] The technical problem or task posed in this technical solution is to create a new effective, simple and reliable method for extracting named entities from a text in Russian. At the same time, the solution should provide high quality extraction of named entities and require as little GPU memory as possible (less than 2 GB). Also, the solution should be able to solve the problem of classifying named entities (at the same time - Joint learning (Bing Liu, Lane Ian. 2016. Attention-based recurrent neural network models for joint intent detection and slot filling.arXiv: 1609.01454) or separately).

[0008] The technical result is to improve the prediction accuracy of named entities.

[0009] The specified technical result is achieved by implementing a method for extracting named entities from textual information, performed by at least one computing device, comprising the steps of:

- receive text information;

- perform the splitting of the text into words;

- tokenize the text to obtain a sequence of tokens;

- a set of vectors is formed by means of a neural network for the received sequence of tokens;

- based on the obtained set of vectors, vector representations of the sequence of tokens are formed;

- by comparing the indicators of the obtained vector representation of the sequence of tokens with predetermined indicators of the vectors obtained as a result of training the neural network, predicting the named entities for the vector representation of the sequence of tokens;

- Recognize the named entities obtained at the previous stage by selecting a word label.

[0010] In one particular embodiment of the method, a vector representation of a sequence of tokens based on a set of vectors is generated by calculating a weighted sum or average values of the vectors of the vectors contained in the set of vectors.

[0011] In another particular embodiment of the method, the step is additionally performed, at which the dimension of the vectors contained in the obtained vector representation of the sequence of tokens is corrected to obtain a vector representation of the sequence of tokens with a given dimension.

[0012] In another particular embodiment of the method, the steps are additionally performed, in which:

- based on the indicators of the vectors contained in the vector representation of the sequence of tokens, and the indicators of the previous vectors in the said sequence, the dependences of the indicators of the vectors of tokens are determined by means of the recurrent layer of the neural network;

- add information about the dependencies of the indicators of the vectors of tokens to the representation of the sequence of tokens by forming a new vector representation of the sequence of tokens.

[0013] In another particular embodiment of the method, the steps are additionally performed, in which:

- for each vector in the vector representations of the sequence of tokens, the dependence of its indicators on the indicators of other vectors is determined;

- based on the indicators of the vector representation of the sequence of tokens and data on the dependencies of the indicators of the vectors, determined at the previous stage, a new vector representation of the sequence of tokens is generated.

[0014] In another particular embodiment of the method, the final word mark can be determined by means of neural networks by voting.

[0015] In another particular embodiment of the method, the steps are additionally performed, in which:

- based on the indicators of the vector representation of the sequence of tokens, the maximum values of these indicators for all vectors are determined and on their basis a vector is formed containing the maximum values of the indicators of the obtained vector representation of the sequence of tokens;

- based on the indicators of the vector representation of the sequence of tokens, the average values of these indicators for all vectors are determined and on their basis a vector is formed containing the average values of the indicators of the obtained vector representation of the sequence of tokens;

- based on the indicators of the vector containing the maximum values of the indicators of the vector representation of the sequence of tokens, the vector containing the average values of the indicators of the obtained vector representation of the sequence of tokens, and the indicators of the last vector in the vector representation of the sequence of tokens, form the resulting vector;

- carry out the classification of the resulting vector to determine the class of the vector representation of the sequence of tokens.

[0016] In another preferred embodiment of the claimed solution, a named entity retrieval system is provided comprising at least one computing device and at least one memory device containing machine-readable instructions that, when executed by at least one computing device, perform the above method.

BRIEF DESCRIPTION OF DRAWINGS

[0017] The features and advantages of the present technical solution will become apparent from the following detailed description of the invention and the accompanying drawings, in which:

[0018] in FIG. 1 shows the general scheme of interaction between the elements of the named entity extraction system;

[0019] in FIG. 2 shows an example of a general view of the named entity extraction system.

CARRYING OUT THE INVENTION

[0020] The following will describe the concepts and terms necessary to understand this technical solution.

[0021] In this technical solution, the system means, inter alia, a computer system, a computer (electronic computer), a CNC (numerical control), a PLC (programmable logic controller), computerized control systems and any other devices capable of performing a given, well-defined sequence of operations (actions, instructions).

[0022] By a command processing device is meant an electronic unit, a computing device, or an integrated circuit (microprocessor) that executes machine instructions (programs).

[0023] An instruction processor reads and executes machine instructions (programs) from one or more storage devices. The role of data storage devices can be, but are not limited to, hard drives (HDD), flash memory, ROM (read only memory), solid state drives (SSD), optical drives.

[0024] A program is a sequence of instructions for execution by a computer control device or command processing device.

[0025] Database (DB) - a collection of data organized in accordance with a conceptual structure describing the characteristics of this data and the relationship between them, and such a collection of data that supports one or more areas of application (ISO / IEC 2382: 2015, 2121423 " database ").

[0026] Entity - any distinguishable object (an object that we can distinguish from another), information about which needs to be stored in a database.

[0027] An entity instance is a specific representative of a given entity. For example, a representative of the "Employee" entity can be "Employee Ivanov". Entity instances must be distinguishable, i.e. entities must have some properties that are unique to each instance of that entity.

[0028] A named entity is an object of a specific type that has a name, title, or identifier. What types the system distinguishes is determined within the framework of a specific task. For a news domain, these are usually people (PER), places (LOC), organizations (ORG) and miscellaneous, objects of a wide range: events, slogans, etc.

[0029] In accordance with the diagram shown in Fig. 1, the named entity retrieval system 100 comprises interconnected: a text preprocessing and tokenizing unit 10; module 20 generating vectors; module 30 for generating a vector representation of a sequence of tokens; module 40 for determining the dependence of the indicators of the vectors of tokens; module 50 for determining the types of dependencies between tokens in the sequence of vector representation of tokens; module 60 for adjusting the dimension of vectors; module 70 predicting named entities for tokens; module 80 predicting the class of the vector representation of the tokens; and a named entity recognition module 90.

[0030] Text preprocessing and tokenization module 10 may be implemented on at least one computing device equipped with appropriate software and include a set of models for text tokenization, for example, a set of WordPiece models (see Yonghui Wu, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Mike Schuster, Zhifeng Chen.2016. Googles neural machine translation system: Bridging the gap between human and machine translation, volume arXiv: 1609.08144).

[0031] The vector generating unit 20 may be implemented based on the BERT language model. BERT is a neural network from Google that has shown by a wide margin state-of-the-art results on a number of problems. With BERT, you can create AI programs for natural language processing, answer freeform questions, create chat bots, automatic translators, analyze text, and more.

[0032] The module 30 for generating a vector representation of a sequence of tokens can be implemented on the basis of at least one computing device equipped with appropriate software for determining a weighted sum or average (depending on the configuration) values of the vector indicators for generating a vector representation of a sequence of tokens.

[0033] The module 40 for determining the dependence of the indicators of the vectors of tokens can be implemented on the basis of a recurrent neural network (RNN), made with the ability, based on the values of the indicators of the vectors contained in the vector representation of the sequence of tokens, to determine recurrent relationships between the indicators of the vectors. The graph of computations of recurrent neural networks can also contain loops that reflect the dependence of the value of a variable at the next moment in time on its current value.

[0034] Module 50 for determining the types of dependencies between tokens in the sequence of vector representation of tokens can be implemented on the basis of a neural network containing a Multi-Head attention neural network layer. Multi-head attention is a special layer that allows each input vector to interact with other words through the attention mechanism, instead of passing "hidden state" parameters like in RNN or neighboring words like in CNN. It is given vectors Query as input, and several pairs of Key and Value (in practice, Key and Value are always the same vector). Each of them is transformed by a trained linear transformation, and then the scalar product Q is calculated with all K in turn, the result of these scalar products is run through softmax, and with the resulting weights, all vectors are summed into a single vector.

[0035] The module 60 for adjusting the dimension of the vector representation of the sequence of tokens can be implemented on the basis of linear layers of the neural network, designed to process the indicators of vectors to adjust their dimensions. The weighting factors for the line layers can be specified by the designer of the module 60 during its design.

[0036] The module 70 predicting named entities for tokens can be implemented on the basis of at least one neural network, pre-trained on specific sets of named entities and their corresponding sets of vectors. In particular, the softmax layer, widely known from the prior art, can be used to predict named entities (see, for example, https://ru.wikipedia.org/wiki/Softmax).

[0037] The module 80 predicting the class of vector representation of tokens can be implemented on the basis of at least one neural network configured to perform subsampling operations on the maximum (Max Pooling) and average (Average Pooling) values containing at least one linear layer for vector dimension adjustments and softmax layer.

[0038] Named entity recognition module 90 may be implemented based on at least one computing device configured to convert predicted named entity data into words. Said transformation algorithm may be predetermined by the developer of said module 90 during its design.

[0039] At the first stage of the operation of the system 100 for extracting named entities, the text preprocessing and tokenization module 10 receives text, for example, from the data source 1. The data source 1 can be at least one database (DB), which contains text information, or the data source 1 can be a user's device, for example, a laptop or desktop computer, a mobile phone or a smartphone, a tablet, etc.

[0040] Text preprocessing and tokenization unit 10 performs word splitting by space between words and tokenization using the WordPiece model to obtain a sequence of tokens. For example, for the text "I want to issue a debit card" the sequence of tokens will be: X ',' ## ochu ',' o ',' ## fοo ',' ## mi ',' ## th ',' de ', '## bet', '## ovoy', 'cards', '## y'. Also, said module 10 adds a special token "[CLS]" to the beginning of the token sequence.

[0041] Accordingly, the generated sequence of tokens by the further mentioned module 10 is transferred to the module 20 of the formation of vectors, which is known from the prior art methods (see, for example, the article by Anton Emelyanov, Ekaterina Artemova. Gapping parsing using pretrained embeddings, attention mechanism and NCRF. 2019 Computational Linguistics and Intellectual Technologies Papers from the Annual International Conference "Dialogue" (2019) Issue 18. Supplementary volume Pages 21-30 URL: http://www.dialog-21.ru/media/4870/ - dialog2019scopusvolplus.pdf) generates a set of vectors for a sequence of tokens, for example, 12 vector representations of a sequence of tokens, after which the resulting set of vectors are sent by said module 20 to a module 30 for generating a vector representation of a sequence of tokens. Based on the obtained set of vectors, the module 30 generates a vector representation of the token sequence, for example, by calculating the weighted sum or average values of the indicators 12 of the aforementioned vector representations of the token sequence.

[0042] Next, the vector representation of the token sequence obtained by the module 30 is sent to the dimension adjustment module 60, which adjusts the dimension of the vectors contained in the obtained vector representation of the token sequence, for example, by multiplying the vector exponents by the generated random non-integer number, to obtain a vector representation of the token sequence with given dimension.

[0043] In an alternative embodiment of the claimed solution, the obtained vector representation of the sequence of tokens by said module 30 can be sent to the module 40 for determining the dependence of the indicators of the token vectors, which, based on the indicators of the vectors contained in the vector representation of the sequence of tokens, and the indicators of the previous vectors in the said sequence, by means of a recurrent layer determines the dependencies of the indicators of the vectors of tokens, which can be expressed in the form of vectors. The information on the dependencies of the indicators of the token vectors is added by the mentioned module 40 to the vector representation of the token sequence by forming a new vector representation of the token sequence, after which the vector representation of the token sequence generated by the module 40 with information on the dependencies of the indicators of the token vectors is sent to the module 50 for determining the types of dependencies between tokens, or into the module 60 for adjusting the dimension of the vector representation of the sequence of tokens for its processing as described earlier.

[0044] The module 50 for determining the types of dependencies between tokens when obtaining a vector representation of a sequence of tokens for each vector by methods known from the prior art determines the dependence of its indicators on the indicators of other vectors included in the vector representation of the sequence of tokens, which can also be expressed as vectors, after which the said Module 50 corrects the vector representation of the token sequence by generating a new vector representation of the token sequence based on the indicators of the vector representation of the token sequence and data on the dependencies of the vector indicators. Data on dependencies of vector indicators characterize the types of dependencies between tokens contained in the vector representation of a sequence of tokens, and may indicate the presence of a semantic or morphological relationship between tokens. Accordingly, the generated vector representation of the sequence of tokens by the module 50 is sent to the module 60 for adjusting the dimension of the vector representation of the sequence of tokens for processing in the previously described manner.

[0045] After the vector representation of the token sequence with a given dimension has been generated by said module 60, it is sent to the named entity prediction module 70 for tokens. Module 70 predicting named entities for tokens based on comparing the indicators of the obtained vector representation of the sequence of tokens with predetermined indicators of vectors obtained as a result of training the neural network, predicts named entities for the vector representation of the sequence of tokens. For example, for the word "debit", the following sequence of named entity labels in 10 markup can be defined: 'I_DEB_CARD', 'I_DEB_CARD', 'I_Ο'. The "O" label means that there is no entity (in this case, the model was mistaken on one token). The final named entity prediction for the word "debit" will be "DEB_CARD".

[0046] Next, the data on the named entities by said module 70 is sent to the named entity recognition module 90, which selects a word label for each named entity using methods known from the prior art, and the final word label can be determined by means of neural networks by voting, and as the final the one with the largest number of labels is selected. For example, 2 votes can be defined for the "I_DEB_CARD" tag, and 1 vote for the "I_O" tag. Accordingly, the final named entity label prediction for the word will also be "DEB_CARD".

[0047] Also additionally, the vector representation of the token sequence with a given dimension can be processed by the unit 80 to predict the class of the vector representation of the token sequence. The mentioned module 80, based on the indicators of the vector representation of a sequence of tokens with a given dimension, obtained from the previously mentioned module 60, using the Max Pooling operation determines the maximum values of these indicators for all vectors and, on their basis, forms a vector containing the maximum values of the indicators of the obtained vector representation of the sequence of tokens ... Also, the mentioned module 80, based on the indicators of the vector representation of a sequence of tokens with a given dimension using the Average Pooling operation, determines the average values of these indicators for all vectors and, on their basis, forms a vector containing the average values of the indicators of the obtained vector representation of the sequence of tokens, after which, based on the indicators of the vector containing the maximum values of the indicators of the vector representation of the sequence of tokens, the vector containing the average values of the indicators of the obtained vector representation of the sequence of tokens, and the indicators of the last vector in the vector representation of the sequence of tokens, said module 80 generates the resulting vector, for example, by means of a concatenation operation. Further, the resulting vector is processed by a linear layer, which is part of module 80, to correct its dimension in the manner described earlier, and a softmax layer to classify the resulting vector and determine, based on the classification result, a class for the vector representation of a sequence of tokens for which the resulting vector was generated.

[0048] The class information of the vector representation of the token sequence may indicate, for example, the presence or absence of named entities in the textual information or the type of user request (intent) contained in the textual information for which said vector representation of the token sequence has been generated. For example, if the text information is the text “there were transactions on my debit card numbered xxxx xxxx xxxx xxxx that I did not do. What to do? ", Which will be presented as" О О B_DEB_CARD I_DEB_CARD О B_NUM_CARD I_NUM_CARD О О О О О О О О О О О О О О О О О О О О О B_DEB_CARD I_DEB_CARD О О О О О О О », then the vector representation of the sequence of tokens generated for this text information, module 80 will define the class "Call_operator", indicating the type of user request - operator call. If the text information will represent the text "I want a debit card", then the class will be defined as "want_card", indicating the type of request - card design. Also, the type of user request may indicate a desire to receive a loan or other service.

[0049] The word label data by the named entity recognition module 90 and the classification data of the vector representation of the token sequence by the vector token sequence class prediction module 80 can then be sent to the storage device 2 for subsequent display to the user upon request.

[0050] Thus, due to the fact that the vector representation of the sequence of tokens is formed on the basis of a set of vectors obtained by the neural network as a result of processing the data of the sequence of tokens, and the prediction of named entities for the vector representation of the sequence of tokens is performed by comparing the indicators of the obtained vector representation of the sequence of tokens with predetermined indicators of vectors obtained as a result of training a neural network, the accuracy of prediction of named entities is increased. Also, in addition, the accuracy of prediction of named entities is ensured due to the fact that the prediction takes into account information about the dependences of the indicators of token vectors on other vectors contained in the vector representation of a sequence of tokens.

[0051] In general (see Fig. 2), the system (200) for extracting named entities contains one or more processors (201) united by a common data exchange bus, memory means such as RAM (202) and ROM (203), interfaces input / output (204), input / output (205), and a device for networking (206).

[0052] The processor (201) (or multiple processors, multi-core processor, etc.) can be selected from a range of devices currently widely used, for example, such manufacturers as: Intel ™, AMD ™, Apple ™, Samsung Exynos ™, MediaTEK ™, Qualcomm Snapdragon ™, etc. Under the processor or one of the processors used in the system (200), it is also necessary to take into account a graphics processor, for example, an NVIDIA GPU with a CUDA compliant programming model, or Graphcore, the type of which is also suitable for full or partial execution of the method, and can also be used for training and application of machine learning models in various information systems.

[0053] RAM (202) is a random access memory and is intended for storing machine-readable instructions executed by the processor (201) for performing the necessary operations for logical data processing. RAM (202), as a rule, contains executable instructions of the operating system and corresponding software components (applications, software modules, etc.). In this case, the available memory of the graphics card or graphics processor can act as RAM (202).

[0054] ROM (203) is one or more persistent storage devices such as a hard disk drive (HDD), solid state data storage device (SSD), flash memory (EEPROM, NAND, etc.), optical storage media ( CD-R / RW, DVD-R / RW, BlueRay Disc, MD), etc.

[0055] Various types of I / O interfaces (204) are used to organize the operation of the system components (200) and to organize the operation of external connected devices. The choice of the appropriate interfaces depends on the specific version of the computing device, which can be, but are not limited to: PCI, AGP, PS / 2, IrDa, FireWire, LPT, COM, SATA, IDE, Lightning, USB (2.0, 3.0, 3.1, micro, mini, type C), TRS / Audio jack (2.5, 3.5, 6.35), HDMI, DVI, VGA, Display Port, RJ45, RS232, etc.

[0056] To ensure user interaction with the computing system (200), various means (205) I / O information are used, for example, a keyboard, display (monitor), touch display, touch-pad, joystick, mouse manipulator, light pen, stylus, touch panel, trackball, speakers, microphone, augmented reality, optical sensors, tablet, light indicators, projector, camera, biometric identification (retina scanner, fingerprint scanner, voice recognition module), etc.

[0057] The means of networking (206) provides data transmission via an internal or external computer network, for example, Intranet, Internet, LAN, etc. One or more means (206) may be used, but not limited to: Ethernet card, GSM modem, GPRS modem, LTE modem, 5G modem, satellite communication module, NFC module, Bluetooth and / or BLE module, Wi-Fi module, etc.

[0058] Additionally, satellite navigation means can be used as part of the system (200), for example, GPS, GLONASS, BeiDou, Galileo.

[0059] The specific choice of elements of the system (200) for the implementation of various software and hardware architectural solutions can vary while maintaining the required functionality provided.

[0060] Modifications and improvements to the above-described embodiments of the present technical solution will be apparent to those skilled in the art. The foregoing description is provided by way of example only and is not intended to be limiting in any way. Thus, the scope of the present technical solution is limited only by the scope of the attached claims.

Claims (23)

1. A method for extracting named entities from textual information, performed by at least one computing device, comprising the steps of: - receive text information; - perform the splitting of the text into words; - tokenize the text to obtain a sequence of tokens; - a set of vectors is formed by means of a neural network for the received sequence of tokens; - based on the obtained set of vectors, a vector representation of a sequence of tokens is formed; - by comparing the indicators of the obtained vector representation of the sequence of tokens with predetermined indicators of the vectors obtained as a result of training the neural network, predicting the named entities for the vector representation of the sequence of tokens; - Recognize the named entities obtained at the previous stage by selecting a word label. 2. The method according to claim 1, characterized in that the vector representation of the sequence of tokens based on the set of vectors is generated by calculating the weighted sum or average values of the indicators of the vectors contained in the set of vectors. 3. The method according to claim 1, characterized in that it further comprises the step of correcting the dimension of the vectors contained in the obtained vector representation of the sequence of tokens to obtain a vector representation of the sequence of tokens with a given dimension. 4. The method according to claim 1, characterized in that it further comprises the stages, at which: - based on the indicators of the vectors contained in the vector representation of the sequence of tokens, and the indicators of the previous vectors in the said sequence, the dependences of the indicators of the vectors of tokens are determined by means of the recurrent layer of the neural network; - add information about the dependencies of the indicators of the vectors of tokens to the representation of the sequence of tokens by forming a new vector representation of the sequence of tokens. 5. The method according to claim 1, characterized in that it further comprises the stages at which: - for each vector in the vector representation of the sequence of tokens, the dependence of its indicators on the indicators of other vectors is determined; - based on the indicators of the vector representation of the sequence of tokens and data on the dependencies of the indicators of the vectors defined in the previous step, a new vector representation of the sequence of tokens is generated. 6. The method according to claim 1, characterized in that the final word mark can be determined by means of neural networks by the voting method. 7. The method according to claim 1, characterized in that it further comprises the stages at which: - based on the indicators of the vector representation of the sequence of tokens, the maximum values of these indicators for all vectors are determined and on their basis a vector is formed containing the maximum values of the indicators of the obtained vector representation of the sequence of tokens; - based on the indicators of the vector representation of the sequence of tokens, the average values of these indicators for all vectors are determined and on their basis a vector is formed containing the average values of the indicators of the obtained vector representation of the sequence of tokens; - based on the indicators of the vector containing the maximum values of the indicators of the vector representation of the sequence of tokens, the vector containing the average values of the indicators of the obtained vector representation of the sequence of tokens, and the indicators of the last vector in the vector representation of the sequence of tokens, form the resulting vector; - carry out the classification of the resulting vector to determine the class of the vector representation of the sequence of tokens. 8. A system for extracting named entities, comprising at least one computing device and at least one memory device containing machine-readable instructions, which, when executed by at least one computing device, perform the method according to any one of claims. 1-7.

Images