KR20210147862A - Method and apparatus for training retrosynthesis prediction model - Google Patents

Abstract

A method for learning of an inverse synthesis prediction model comprises: a step of determining the information to be paid attention to in a first character string information of the product based on the first graph information of the product, and encoding the first character string information based on a determination result; a step of determining the information to be attended to from the first graph information and the second graph information of a reactant, and decoding the second character string information of the reactant based on the determination result; and a step of learning the inverse synthesis prediction model based on a decoding result of the second character string information.

Description

Method and apparatus for training retrosynthesis prediction model

It relates to a learning method and apparatus for a retrosynthesis prediction model.

Neural network refers to a computational architecture that models the biological brain. As neural network technology develops, various types of electronic systems use neural networks to analyze input data and extract valid information.

A technology capable of accurately predicting a reactant from a product using a neural network is required.

An object of the present invention is to provide a learning method and apparatus for a retrosynthesis prediction model. The technical problem to be achieved by the present embodiment is not limited to the above technical problems, and further technical problems may be inferred from the following embodiments.

According to one aspect, the method for learning the inverse synthesis prediction model determines first attention information from the first character string information of the product based on first graph information of the product, and based on the determination result, the first Encoding string information, determining second attention information from the first graph information and second graph information of the reactant, and decoding second string information of the reactant based on the determination result, and the second string information and training the inverse synthesis prediction model based on the decoding result of .

According to another aspect, the computer-readable recording medium includes a recording medium in which one or more programs including instructions for executing the above-described method are recorded.

According to another aspect, an apparatus for predicting a reaction product using the inverse synthesis prediction model includes a memory storing at least one program and a processor executing the at least one program, wherein the processor includes first graph information of the product Determines first attention information from the first character string information of the product based on The second attention information is determined, the second string information of the reactant is decoded based on the determination result, and the inverse synthesis prediction model is trained based on the decoding result of the second string information.

1 is a block diagram illustrating a hardware configuration of a neural network device according to an embodiment.
2 is a diagram for explaining character string information according to an embodiment.
3 is a diagram for explaining graph information according to an embodiment.
4 is a diagram for explaining a method of learning an inverse synthesis prediction model performed by a processor according to an embodiment.
5 is a diagram for explaining an encoding method according to an embodiment.
6 is a diagram for describing a method of generating a first mask matrix according to an exemplary embodiment.
7 is a diagram referenced to describe a method of generating a first mask matrix according to an embodiment.
8 is a diagram for describing an effect of a first mask matrix according to an exemplary embodiment.
9 is a diagram for explaining a decoding method according to an embodiment.
10 is a diagram for describing a method of generating a second mask matrix according to an exemplary embodiment.
11 is a diagram referenced to describe a method of generating a second mask matrix according to an embodiment.
12 is a flowchart illustrating a method of operating an inverse synthesis prediction model according to an embodiment.
13 is a flowchart illustrating an encoding method according to an embodiment.
14 is a flowchart illustrating a decoding method according to an embodiment.
15 is a flowchart illustrating a method of learning an inverse synthesis prediction model according to an embodiment.

The appearances of the phrases "in some embodiments" or "in one embodiment" in various places in this specification are not necessarily all referring to the same embodiment.

Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented in various numbers of hardware and/or software configurations that perform specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors, or by circuit configurations for a given function. Also, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented as an algorithm running on one or more processors. In addition, the present disclosure may employ prior art for electronic configuration, signal processing, and/or data processing, and the like. Terms such as “mechanism”, “element”, “means” and “configuration” may be used broadly and are not limited to mechanical and physical components.

In addition, the connecting lines or connecting members between the components shown in the drawings only exemplify functional connections and/or physical or circuit connections. In an actual device, a connection between components may be represented by various functional connections, physical connections, or circuit connections that are replaceable or added.

Meanwhile, in relation to the terms used in this specification, a structure that is data used in a neural network system may mean an atom-level structure of a material. The structure may be a structural formula based on a bond between atoms.

1 is a block diagram illustrating a hardware configuration of a neural network device according to an embodiment.

The neural network apparatus 100 may be implemented with various types of devices such as a personal computer (PC), a server device, a mobile device, and an embedded device, and as a specific example, voice recognition, image recognition, image classification, etc. using a neural network It may correspond to a smartphone, tablet device, AR (Augmented Reality) device, IoT (Internet of Things) device, autonomous vehicle, robotics, medical device, etc., but is not limited thereto. Furthermore, the neural network apparatus 100 may correspond to a dedicated hardware accelerator mounted on the above device, and the neural network apparatus 100 is a dedicated module for driving a neural network, a neural processing unit (NPU), It may be a hardware accelerator such as a Tensor Processing Unit (TPU), a Neural Engine, or the like, but is not limited thereto.

Referring to FIG. 1 , the neural network device 100 includes a processor 110 , a memory 120 , and a user interface 130 . In the neural network device 100 shown in FIG. 1, only the components related to the present embodiments are shown. Accordingly, it is apparent to those skilled in the art that the neural network device 100 may further include other general-purpose components in addition to the components shown in FIG. 1 .

The processor 110 serves to control overall functions for executing the neural network device 100 . For example, the processor 110 generally controls the neural network device 100 by executing programs stored in the memory 120 in the neural network device 100 . The processor 110 may be implemented as a central processing unit (CPU), a graphics processing unit (GPU), an application processor (AP), etc. included in the neural network device 100 , but is not limited thereto.

The memory 120 is hardware for storing various data processed in the neural network device 100 . For example, the memory 120 may store data processed by the neural network device 100 and data to be processed. have. Also, the memory 120 may store applications, drivers, and the like to be driven by the neural network device 100 . The memory 120 includes random access memory (RAM), such as dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD- It may include ROM, Blu-ray or other optical disk storage, a hard disk drive (HDD), a solid state drive (SSD), or flash memory.

The processor 110 may train a retrosynthesis prediction model through a test data set, and predict at least one combination of reactants from a target product based on the learned retrosynthesis prediction model. Reverse synthesis prediction may mean predicting a combination of reactant molecules for synthesizing a target product by exploring an inverse reaction pathway of the target product.

The inverse synthesis prediction model may be implemented using a transformer model. As the neural network device 100 uses the transformer model, parallel processing of data and rapid computation are possible.

The processor 110 may train the transformer model using the test data set.

The test data set may include test products and combinations of test reactants corresponding to each of the test products. For example, any one of the test products may be benzene, and the test reactant combination corresponding to benzene may be furan and ethylene.

According to an embodiment, the test data set may further include a predicted yield of test products according to an experimental condition and reaction methods of each of test reactant combinations.

Experimental conditions may refer to various conditions set in order to proceed with an experiment for generating a product using a reactant. For example, the experimental conditions may include at least one of a catalyst, a base, a solvent, a reagent, a temperature, and a reaction time.

Reaction mode may refer to a chemical reaction method for producing a product using reactant combinations.

The processor 110 may receive test products and a test reactant combination corresponding to each of the test products to train the inverse synthesis prediction model.

The processor 110 may receive the chemical structure of the test product in a one-dimensional string format and a two-dimensional graph format. In addition, the processor 110 may receive the chemical structure of the test reactant in a one-dimensional string format and a two-dimensional graph format.

The processor 110 may encode the first string information of the test product in a vector format.

The processor 110 may determine information to be focused on in the first string information based on the first graph information of the test product, and encode the first string information based on the determination result. Information to be focused on in the first character string information may be referred to as first attention information. The first attention information may be information on neighboring atoms adjacent to the encoding target atom. In other words, the processor 110 obtains information on adjacent atoms of the encoding object atom from the first graph information of the test product, and further pays attention to the relationship between the encoding object atom and the adjacent atoms to encode the encoding object atom can do.

As the processor 110 specifies information related to input information (eg, token) from the entire string information, and encodes the input information by paying attention to the specified information, it occurs as all string information is encoded into a fixed-length vector information vanishing can be reduced. In addition, as the processor 110 encodes the encoding target atom by paying more attention to the relationship between the encoding target atom and the chemically related adjacent atoms, efficient and rapid learning of the inverse synthesis prediction model is possible, The accuracy of the synthetic prediction model can be significantly increased.

The processor 110 may output the encoded first string information as a first output sequence.

The processor 110 may determine information to be paid attention to in the first graph information of the test product and the second graph information of the test reactant, and decode the second string information of the reactant based on the determination result. Information to be paid attention to in the first graph information and the second graph information may be referred to as second attention information. The relationship between the test product and the test reactant can be represented by a cross attention matrix, and the ideal cross attention matrix is atom-mapping information representing the relationship between the atoms included in the product and the atoms included in the reactant. to catch Accordingly, information to be paid attention to in the first graph information and the second graph information for learning the inverse synthesis prediction model may be atomic mapping information. In other words, the processor 110 may obtain atomic mapping information from the first graph information and the second graph information, and further pay attention to the atomic mapping information to decode an atom to be decoded.

When the processor 110 predicts output information (eg, token), it specifies a highly relevant part (eg, token) in the input string information, and decodes the input information by paying attention to the specified information , information loss (vanishing) due to the lengthening of the input string can be reduced.

In addition, as the processor 110 decodes an atom to be decoded with more attention to the atomic mapping information for learning the inverse synthesis prediction model, efficient and rapid learning of the inverse synthesis prediction model is possible, and the accuracy of the inverse synthesis prediction model can be significantly increased.

The processor 110 may train the inverse synthesis prediction model based on the decoding result of the second string information. That the processor 110 trains the inverse synthesis prediction model may mean that the cross-attention matrix learns hidden states of the encoder and the decoder to follow the atomic mapping information. In addition, the meaning that the processor 110 trains the inverse synthesis prediction model may mean that it trains information on the edge indicating the relationship between the node and the node in the graph information of each test product and test reactant. have. Also, the meaning that the processor 110 trains the inverse synthesis prediction model may mean that the accuracy of the prediction result of the neural network apparatus 100 is improved. According to an embodiment, the fact that the processor 110 trains the inverse synthesis prediction model may mean that it learns atomic mapping information indicating a relationship between an atom included in a product and an atom included in a reactant.

The processor 110 may predict at least one candidate reactant combination from the input product based on the learned inverse synthesis prediction model. Since the inverse synthesis prediction model is implemented using the transformer model, the processor 110 may predict a candidate reactant combination from the input product using the transformer model.

The user interface 130 may refer to an input means for receiving a feedback of an experiment result. For example, the user interface includes a key pad, a dome switch, and a touch pad (contact capacitive method, pressure resistive film method, infrared sensing method, surface ultrasonic conduction method, integral tension measurement method) , piezo effect method, etc.), a jog wheel, a jog switch, etc., but is not limited thereto. The processor 110 may update the transformer model by receiving a feedback of the experimental result.

Hereinafter, in the description of the present disclosure, a method of learning the inverse synthesis prediction model will be described in detail. The methods described below may be performed by the processor 110 , the memory 120 , and the user interface 130 of the neural network device 100 . In addition, for convenience of description, hereinafter, a product refers to a test product, and a reactant refers to a test reactant.

2 is a diagram for explaining character string information according to an embodiment.

Referring to FIG. 2 , chemical structures of products and reactants may be input to the neural network device 100 in the form of a one-dimensional character string. The structure may be a structural formula based on a bond between atoms. In an embodiment, the processor 110 may receive chemical structures of products and reactants in a Simplified Molecular-Input Line-Entry System (SMILES) format.

Since the SMILES notation is not unique and depends on the choice of the central atom or the start of the sequence, a standardized algorithm can be used. The SMILES notation of this disclosure separates each atomic token (eg, B, C, N, O) and a non-atom token. A non-atomic token may contain bonds between atoms (eg, -, =, #), parentheses, a number of cyclic structures with whitespace, and the like.

For example, when furan 210 and ethylene 220 combine to produce benzene 230, the furan 210, ethylene 220, and benzene 230 SMILES are c1cocc1, c=c and c1ccccc1 can be expressed as

Although only the SMILES notation is shown in FIG. 2 , according to an embodiment, the processor 110 may receive chemical structures of products and reactants in the SMARTS (Smiles Arbitrary Target Specification) format or InChi (International Chemical Identifier) format. It is also possible .

3 is a diagram for explaining graph information according to an embodiment.

Referring to FIG. 3 , chemical structures of products and reactants may be input to the neural network device 100 in the form of a two-dimensional graph.

Graph information may include nodes and edges. A node may include information about atoms of products and reactants, and an edge may include information about a connection relationship between each atom.

The transformer model encodes and decodes string information, but it can also be reinterpreted as a graph neural network. In an embodiment, tokens of a source sequence and a target sequence input to the transformer model may correspond to nodes. In addition, the attention (attention) of the transformer model may correspond to the edge. For example, the graph information 310 of the benzene 230 may include first to sixth nodes n1 to n6 . In addition, the graph information 310 may include first to sixth edges e1 to e6 indicating the connection relationship of each node. Although the value of the edge is not known initially, it can be grasped by learning the transformer model.

4 is a diagram for explaining a method of learning an inverse synthesis prediction model performed by a processor according to an embodiment.

Referring to FIG. 4 , the processor 110 trains a reverse synthesis prediction model through a test data set, and predicts at least one reactant combination from a target product based on the learned reverse synthesis prediction model. have. To this end, the processor 110 includes the encoder embedding unit 410 , the first position encoding unit 420 , the encoder unit 430 , the first mask matrix generation unit 440 , the decoder embedding unit 450 , and the second position It may include an encoder 460 , a decoder 470 , and a second mask matrix generator 480 . In FIG. 4 , the encoder embedding unit 410 , the first position encoding unit 420 , the encoder unit 430 , the first mask matrix generation unit 440 , the decoder embedding unit 450 , and the second position encoding unit 460 . , the decoder unit 470 and the second mask matrix generation unit 480 are shown as any one unit included in the processor 110 , but the encoder embedding unit 410 and the first position encoding unit 420 are ), the encoder unit 430 , the first mask matrix generation unit 440 , the decoder embedding unit 450 , the second position encoding unit 460 , the decoder unit 470 , and the second mask matrix generation unit 480 are It may mean a layer included in the transformer model of the processor 110 . In addition, a residual connection sub-layer and a normalization sub-layer may be individually applied to all lower layers.

The encoder embedding unit 410 may embed input data in character units. In this case, the input data may mean the first character string information 491 of the product. In other words, the encoder embedding unit 410 may map the first string information 491 in a string format to a vector having a preset dimension. For example, the preset dimension may be 512 dimensions, but is not limited thereto.

The first positional encoding unit 420 may perform positional encoding to identify the position of each character included in the first string information 491 . For example, the first position encoding unit 420 may position-encode the first character string information 491 using sine waves of different frequencies. The first position encoding unit 420 may provide the encoder unit 430 by combining the position information with the embedded first character string information 491 .

The encoder unit 430 may include an encoder self-attention unit 431 and an encoder feed forward unit 432 . In FIG. 4 , the encoder unit 430 is illustrated as one unit, but N encoders may be stacked. In addition, in FIG. 4, the encoder self-attention unit 431 and the encoder feed-forward unit 432 are displayed as one unit, the encoder unit 430, but the encoder self-attention unit 431 and the encoder feed-forward unit ( 432 may mean each sub-layer included in the encoder layer.

When encoding the first character string information 491 , the encoder self-attention unit 431 may specify information to be self-attention to in the first character string information 491 . To this end, the encoder self-attention unit 431 may generate a self-attention score matrix. A score indicating a degree of relevance between character strings included in the first character string information 491 may be mapped to the self-attention score matrix. The encoder self-attention unit 431 may generate a self-attention matrix representing element values of the self-attention score matrix as probabilities by using the soft max function.

When encoding the input string based on the self-attention matrix, the encoder unit 430 may specify elements to be self-attended in the first string information 491 . The encoder self-attention unit 431 may be configured as a multi-head to generate a plurality of self-attention matrices.

Meanwhile, the encoder self-attention unit 431 may not adjust the space expressing the degree of relevance between tokens included in the first string information 491 , but self-attention and cross-attention, which will be described later A graph structure that can be applied to attention) can be used to limit the space expressing the degree of relevance.

The processor 110 determines information to be attended when encoding the first string information 491 in the self-attention score matrix and/or the self-attention matrix based on the first graph information 441 for efficiency, speed, and accuracy of learning. can do. Information to be attended to in the self-attention score matrix and/or the self-attention matrix may be first attention information. To this end, the first mask matrix generator 440 may generate a first mask matrix 442 for masking unnecessary elements in the self-attention score matrix and/or the self-attention matrix.

The first mask matrix generator 440 may generate a first mask matrix 442 based on the first graph information 441 of the product and a preset reference distance.

The first mask matrix generator 440 may generate a first pre mask matrix by setting tokens included in the first string information 491 as rows and columns. Also, the first mask matrix generator 440 may determine neighboring nodes in the first graph information 441 based on a preset reference distance. Also, the first mask matrix generator 440 may generate the first mask matrix 442 based on the determined neighboring nodes. For example, the first mask matrix generator 440 allocates information on nodes necessary for calculating the self-attention matrix to the first pre-mask matrix, but allocates '1' to the reference node and neighboring nodes, and , by allocating '0' to the remaining nodes, the first mask matrix 442 may be generated. When the first mask matrix generator 440 generates the first mask matrix 442 based on the neighboring nodes of the reference node, the desynthesis prediction model is generated by paying more attention to neighboring nodes that are chemically related to the reference node. in order to learn

The first mask matrix generator 440 may provide the first mask matrix 442 to the encoder self-attention unit 431 .

The encoder self-attention unit 431 may determine an element to be attended to when encoding the first string information 491 in the self-attention matrix based on the first mask matrix 442 . Also, the encoder self-attention unit 431 may output a self-attention matrix to which a mask is applied based on the determination result.

The self-attention matrix to which the mask is applied may be provided to the encoder feed forward unit 432 .

The encoder feed-forward unit 432 may include a feed-forward neural network. The input sequence may be converted and output by the feed-forward neural network. The transformed input sequence may be provided to the decoder unit 470 .

The decoder embedding unit 450 may embed input data in character units. In this case, the input data may mean the second character string information 492 of the reactant. In other words, the decoder embedding unit 450 may map the second string information 492 in a string format to a vector having a preset dimension. For example, the preset dimension may be 512 dimensions, but is not limited thereto.

The second positional encoding unit 460 may perform positional encoding to identify the position of each character included in the second string information 492 . For example, the second position encoding unit 460 may position-encode the second character string information 492 using sine waves of different frequencies. The second position encoding unit 460 may provide the decoder unit 470 by combining the position information with the embedded second character string information 492 .

The decoder unit 470 may include a decoder self-attention unit 471 , a decoder cross-attention unit 472 , and a decoder feed forward unit 473 . Like the encoder unit 430 , the decoder unit 470 is illustrated as one unit, but N decoders may be stacked. In addition, the decoder self-attention unit 471 , the decoder cross-attention unit 472 , and the decoder feed-forward unit 473 are displayed as one unit, but the decoder self-attention unit 471 and the decoder cross-attention unit 472 ) and the decoder feed forward unit 473 may refer to each sub-layer included in the decoder layer.

The decoder self-attention unit 471 may perform an operation similar to that of the encoder self-attention unit 431 . In other words, when the decoder self-attention unit 471 decodes the second string information 492, the second string information 492 specifies information to be self-attention to, and generates a self-attention matrix based on the specified information. can In addition, the decoder self-attention unit 471 may also be configured as a multi-head to generate a plurality of self-attention matrices. In order to distinguish the self-attention matrix generated by the decoder self-attention unit 471 from the self-attention matrix generated by the encoder self-attention unit 431, the self-attention matrix generated by the encoder self-attention unit 431 is first self-attention. A matrix may be named, and the self-attention matrix generated by the decoder self-attention unit 471 may be referred to as a second self-attention matrix.

The difference between the decoder self-attention unit 471 and the encoder self-attention unit 431 is whether a mask is applied. The decoder self-attention unit 471 does not mask the self-attention matrix using the first mask matrix 442 described in the encoder self-attention unit 431 . However, the decoder self-attention unit 471 may use a mask for preventing the current output position from being used as information on the next output position.

When decoding the second character string information 492 , the decoder cross-attention unit 472 may specify information to be cross-attended in the first character string information 491 . To this end, the decoder cross-attention unit 472 may generate a cross attention score matrix indicating the degree of relevance between the second string information 492 and the first string information 491 . A score indicating a degree of relevance between character strings included in the first character string information 491 and the character strings included in the second character string information 492 may be mapped to the cross attention score matrix. The decoder cross-attention unit 472 may use a soft max function to generate a cross-attention matrix representing element values of the cross-attention score matrix as probabilities. The decoder unit 470 may specify, in the first string information 491 , elements to be cross-attended upon when decoding the input string based on the cross-attention matrix.

The processor 110 may determine the information to be attended to when calculating the attention loss in the cross attention matrix based on the first graph information 441 and the second graph information 481 for efficiency, speed, and accuracy of learning. Information to be attended to in the cross attention matrix may be second attention information. To this end, the second mask matrix generator 480 may generate a second mask matrix 482 for masking unnecessary elements in the cross attention matrix. The roles of the first mask matrix 442 and the second mask matrix 482 may be different from each other. Unlike the first mask matrix 442 , the second mask matrix 482 may be a matrix for masking unnecessary elements when an attention loss of the desynthesis prediction model is calculated.

The second mask matrix generator 480 is configured to perform an atom mapping (atom-) representing a correspondence between atoms included in a product and atoms included in a reactant based on the first graph information 441 and the second graph information 481 . mapping) information may be obtained, and a second mask matrix 482 may be generated based on the atomic mapping information.

The second mask matrix generator 480 may set the second string information 492 as rows and set the first string information 491 as columns to generate a second free mask matrix. Also, the second mask matrix generator 480 may obtain atom-mapping information from the first graph information 441 and the second graph information 481 . When a Maximum Common Substructure (MCS) technique is used to determine the similarity between the first graph information 441 and the second graph information 481 , a non-deterministic polynomial time (NP)-hard problem occurs. can be Accordingly, the atomic mapping information of the present disclosure does not require accurate atomic mapping information in order to determine the degree of similarity between the first graph information 441 and the second graph information 481 and uses only a specific pair of information for a flexible maximum common sub It may be configured using a flexible maximum common substructure (FMCS) technique. The flexible maximum common substructure (FMCS) may be the FMCS algorithm implemented in the RDkit. As the second mask matrix generator 480 uses atomic mapping using only a specific pair of information rather than accurate atomic mapping information, computing cost may be reduced.

The second mask matrix generator 480 may generate a second mask matrix 482 based on the obtained atomic mapping information. For example, the second mask matrix generator 480 allocates information on nodes necessary for calculating the attention loss to the second premask matrix, assigns '1' to corresponding nodes, and By assigning '0' to , the second mask matrix 482 may be generated.

The second mask matrix generator 480 may provide the second mask matrix 482 to the decoder cross-attention unit 472 .

The decoder cross-attention unit 472 may determine an element to be attended to when calculating the attention loss of the desynthesis prediction model in the cross-attention matrix based on the second mask matrix 482 . Also, the decoder cross-attention unit 472 may output a cross-attention matrix to which a mask is applied based on the determination result. The masked cross attention matrix may be provided to the decoder feed forward unit 473 .

The decoder feed forward unit 473 may perform an operation similar to that of the encoder feed forward unit 432 . In other words, the decoder feed-forward unit 473 includes a feed-forward neural network, and an input sequence may be converted by the feed-forward neural network. The decoder feed forward unit 473 may output the converted input sequence as the output reactant information 493 .

Meanwhile, the processor 110 may obtain the attention loss of the desynthesis prediction model from the cross-attention matrix to which the mask is applied. Also, the processor 110 may obtain a cross entropy loss of the inverse synthesis prediction model from the output reactant information 493 .

The processor 110 may train the inverse synthesis prediction model based on the loss of attention and the loss of cross entropy. For example, the processor 110 may train the inverse synthesis prediction model so that the sum of the attention loss and the cross entropy loss is small.

FIG. 5 is a diagram for explaining an encoding method according to an embodiment, FIG. 6 is a diagram for explaining a method for generating a first mask matrix according to an embodiment, and FIG. 7 is a diagram for explaining a first mask according to an embodiment Reference is made to a method of generating a matrix, and FIG. 8 is a diagram for explaining an effect of the first mask matrix according to an embodiment.

Referring to FIG. 5 , the encoder self-attention unit 431 may obtain query, key, and value information from the first string information 491 . The encoder self-attention unit 431 may convert an input vector sequence having a first dimension into a query vector, a key vector, and a value vector having a second dimension smaller than the first dimension. The input vector sequence may mean a vector sequence of the first character string information 491 converted by the encoder embedding unit 410 and the first position encoding unit 420 . Dimensional transformation may be performed by multiplying each vector by a weight matrix. The weight matrix can be updated by training. For example, the first dimension may be 512 and the second dimension may be 64, but is not limited thereto.

The encoder self-attention unit 431 may generate a self-attention score matrix 511 based on a query and a key. The encoder self-attention unit 431 may determine the degree of relevance to all keys for each query, and display the determination result in the self-attention score matrix 511 . The self-attention score matrix 511 may include information about a score indicating the relationship between the query and the key. In an embodiment, the self-attention score matrix 511 may be derived by a scaled dot product attention operation as shown in Equation (1).

In Equation 1, S is the self-attention score matrix 511, Q is a query vector, K is a key vector, T is a transpose matrix, and d _k may mean a dimension of the key vector.

The encoder self-attention unit 431 may receive the first mask matrix 442 from the first mask matrix generator 440 .

The first mask matrix generator 440 may generate a first mask matrix 442 for masking unnecessary elements in the self-attention score matrix 511 .

The first mask matrix generator 440 encodes the first graph information 441 based on a preset reference distance in order to learn the inverse synthesis prediction model by paying attention to neighboring nodes that are chemically related to the encoding target node. Neighbors adjacent to the target node may be determined, and a first mask matrix 442 may be generated based on the determination result. In this case, the distance may mean a geodesic distance on the graph. Also, the distance may mean a hop neighbor on the graph.

The first mask matrix generator 440 may set any one node among the nodes included in the first graph information 441 as a reference node. Also, adjacent nodes existing at a distance apart from the reference node by the reference distance may be expressed as “1”, and the remaining nodes may be expressed as “0”. According to an embodiment, the first mask matrix generator 440 may express a reference node and adjacent nodes existing at a distance by a reference distance from the reference node as “1” and the remaining nodes as “0”. have.

6 illustrates a method in which the first mask matrix generator 440 generates the first mask matrix 442 when the reference node is n1 and the reference distance is 1.

Referring to FIG. 6 , the first mask matrix generator 440 may set any one of the nodes n1 to n6 included in the first graph information 441 as the reference node n1 . Also, the first mask matrix generator 440 may determine the neighboring nodes n2 and n6 existing at a distance apart from the first reference node n1 by the reference distance. The first mask matrix generator 440 may allocate the reference node n1 and information on the reference node n1 and neighboring nodes n2 and n6 to the first pre-mask matrix. In an embodiment, the first mask matrix generator 440 allocates “1” to the reference node n1 and neighboring nodes n2 and n6 adjacent to each other, and allocates “0” to the remaining nodes. can 6 illustrates an example in which “1” is assigned to the neighboring nodes n2 and n6, according to an exemplary embodiment, the first mask matrix generator 440 includes the reference node n1 and the reference node n1. “1” may be allocated to the neighboring nodes n2 and n6 that are adjacent to each other, and “0” may be allocated to the remaining nodes.

On the other hand, the first character string information 491 is, in addition to the atomic tokens, bonds between atoms (eg, -, =, #), parentheses, such as the number of cyclic structures having a space (whitespace). Since non-atomic tokens are further included, the tokens of the first string information 491 and the nodes of the first graph information 441 do not match each other. Since these non-atomic tokens can be disambiguated from the full context of string information, a wider range of information may be required. Accordingly, the first mask matrix generator 440 may not mask nodes corresponding to non-atomic tokens. In other words, the first mask matrix generator 440 may assign “1” to nodes corresponding to non-atomic tokens. Since non-atomic tokens exchange attention with all other tokens regardless of the graph structure, the accuracy of the inverse synthesis prediction model can be improved.

The first mask matrix generator 440 may change a reference node and determine neighboring nodes of the changed reference node. Also, the first mask matrix generator 440 may generate a first mask matrix 442 based on the determination result.

When the distance matrix in which information about the reference distance is stored is D=(d _ij ), elements included in the first mask matrix 442 may be determined by Equation 2 below.

In Equation 2, m _ij may mean a value of an element included in the first mask matrix 442 , and i and j may mean an atom token. d _h is a reference distance set to the h-th head, and the first mask matrix generator 440 may set a different reference distance for each head. In other words, the first mask matrix generator 440 may generate a plurality of first mask matrices having different reference distances and provide them to each head included in the encoder self-attention unit 431 .

7 shows an example of a mask matrix according to a reference distance. 7 shows a first mask matrix 711 when the reference distance is 1, a first mask matrix 712 when the reference distance is 2, and a first mask matrix 713 when the reference distance is 3, have.

Referring to FIG. 7 , the first mask matrix generator 440 sets the reference distance of the first mask matrix 711 provided to the first head to 1, and the second mask matrix 712 provided to the second head. ) may be set to 2, and the third mask matrix 713 provided to the third head may be set to 3. As the first mask matrix generator 440 sets the reference distances of the first mask matrices provided to each head to be different from each other, learning of the inverse synthesis prediction model may be strengthened.

Referring back to FIG. 5 , the first mask matrix generator 440 may provide the first mask matrix 442 to the encoder self-attention unit 431 .

The encoder self-attention unit 431 may mask the self-attention score matrix 511 based on the first mask matrix 442 .

When the self-attention score matrix 511 is S=(s _ij ) and the first mask matrix 442 is M=(M _ij ), the self-attention score matrix 511 to which the mask is applied is expressed by Equation 3 and can be expressed as

As in Equation 3, when the element of the first mask matrix 442 is 1 (that is, m _ij =1), the element of the self-attention score matrix 511 is output as it is, and the first mask matrix 442 is When the element of is 0 (that is, m _ij = 0), "-∞" may be output.

The encoder self-attention unit 431 calculates an attention distribution of the self-attention score matrix 511 to which a mask is applied using a softmax function, and weights and sums the calculation result and each value to obtain an attention value. (attention value) can be created. Attention values may be expressed as a self-attention matrix. The score of the self-attention score matrix 511 may be expressed as a probability by the soft max function. Since the self-attention matrix includes context information of a sequence, it may be referred to as a context vector. In other words, the encoder self-attention unit 431 may output a self-attention matrix to which a mask is applied by Equation 4 below.

As the encoder self-attention unit 431 outputs the self-attention matrix from which unnecessary elements are removed, learning efficiency, speed, and accuracy may be improved.

In FIG. 8 , a graph 811 and a first mask matrix 442 are applied to the encoder for explaining the calculation method of the processor 110 when the encoder self-attention is performed without applying the first mask matrix 442 . A graph 813 is shown for explaining the calculation method of the processor 110 in case of self-attention.

Referring to FIG. 8 , the processor 110 removes unnecessary elements in learning the inverse synthesis prediction model and concentrates more on the relationship between chemically related neighboring nodes, without introducing additional parameters , the efficiency, speed and accuracy of learning can be increased.

9 is a diagram illustrating a decoding method according to an embodiment, FIG. 10 is a diagram illustrating a method of generating a second mask matrix according to an embodiment, and FIG. 11 is a second mask according to an embodiment It is a diagram referenced to describe a method of generating a matrix.

Referring to FIG. 9 , the decoder cross-attention unit 471 may obtain key and value information from the first string information 491 provided by the encoder unit 430 . Also, the decoder cross-attention unit 471 may obtain query information from the second string information 492 . A method in which the decoder cross-attention unit 471 obtains the query, key, and value may be similar to a method in which the encoder self-attention unit 431 obtains the query, key and value. In other words, the decoder cross-attention unit 471 may obtain a query, a key, and a value by multiplying an input vector by a weight matrix.

The decoder cross-attention unit 471 may generate a cross-attention score matrix based on the query and the key. The decoder cross-attention unit 471 may determine the degree of relevance to all keys for each query, and display the determination result in a cross-attention score matrix. The cross-attention score matrix may include information about a score indicating a relationship between a query and a key. In an embodiment, the cross attention score matrix may be derived by a scaled dot product attention operation as in Equation 1 above.

The decoder cross-attention unit 471 calculates an attention distribution of a cross-attention score matrix using a softmax function, and weights the calculation result and each value to generate an attention value. can do. Attention values may be expressed as a cross attention matrix 911 .

The second mask matrix generator 480 may generate a second mask matrix 482 for masking unnecessary elements in the cross attention matrix 911 .

Since a reaction is not a process of completely decomposing a molecule to produce a completely new product, the molecule of the product and the molecule of the reactant generally have a common structure. Thus, atomic mapping between the atoms of the product and the atoms of the reactant is possible. Also, since the cross-attention matrix 911 reflects the relationship between the product token and the reactant token, the ideal cross-attention matrix 911 catches the atomic mapping information. Accordingly, the second mask matrix generator 480 may generate the second mask matrix 482 based on the atomic mapping information between the product and the reactant in order to learn the desynthesis prediction model by paying attention to the atomic mapping information. .

The second mask matrix generator 480 may obtain atomic mapping information from the first graph information 441 and the second graph information 481 . The second mask matrix generator 480 may obtain atomic mapping information using a flexible maximum common substructure (FMCS) technique. For example, a flexible maximum common substructure (FMCS) may be, but is not limited to, an FMCS algorithm implemented in RDkit.

The second mask matrix generator 480 may set any one node among the nodes included in the first graph information 441 as a reference node. Also, among the nodes included in the second graph information 481 , a node corresponding to the reference node may be expressed as “1” and the remaining nodes may be expressed as “0”.

10 and 11 illustrate a method in which the second mask matrix generator 480 generates the second mask matrix 482 when the product is benzene and the reactant combination is furan and ethylene.

10 and 11 , the second mask matrix generator 480 may set any one of the nodes n1 to n6 included in the first graph information 441 as the reference node n1. have. In order to distinguish the reference node n1 set by the second mask matrix generator 480 from the reference node set by the first mask matrix generator 440, the reference set by the first mask matrix generator 440 The node may be called a first reference node, and the reference node set by the second mask matrix generator 480 may be called a second reference node.

The second mask matrix generator 480 may determine a node na corresponding to the reference node n1 from among the nodes na to ng of the second graph information 481 . The second mask matrix generator 480 may allocate information on the node na corresponding to the reference node n1 to the second premask matrix. In an embodiment, the second mask matrix generator 480 may allocate “1” to the node na corresponding to the reference node n1 and allocate “0” to the remaining nodes.

In an embodiment, the second mask matrix generator 480 may not mask nodes corresponding to non-atomic tokens as shown in FIG. 10 . In other words, the second mask matrix generator 480 may assign “1” to nodes corresponding to non-atomic tokens. In another embodiment, as shown in FIG. 11 , the second mask matrix generator 480 may mask nodes corresponding to non-atomic tokens in order to pay attention to the correspondence between atoms. In other words, the second mask matrix generator 480 may assign “0” to nodes corresponding to non-atomic tokens.

The second mask matrix generator 480 may change a reference node and determine a node corresponding to the changed reference node. Also, the second mask matrix generator 480 may generate a second mask matrix 482 based on the determination result.

When the reactant is R and the product is P, elements included in the second mask matrix 482 may be determined by Equation 5 below.

In Equation 5, i' is a node index of the second graph information 481 corresponding to the i-th token of the second string information 492, and j' is the j-th token of the first string information 491. It may mean a node index of the first graph information 441 .

Referring back to FIG. 9 , the second mask matrix generator 480 may provide the second mask matrix 482 to the decoder cross-attention unit 471 .

The decoder cross-attention unit 471 may mask the cross-attention matrix 911 based on the second mask matrix 482 .

The masking role of the decoder cross-attention unit 471 may be different from the masking role of the encoder self-attention unit 431 . This is not only because the atomic mapping is not perfect, but also because the auto-regressive nature of the decoder unit 470 in the intersection attention generates incomplete string information (eg, SMILES), and the sequence at the inference time. This is because the decoder unit 470 cannot find the atomic mapping information during generation. Accordingly, the decoder cross-attention unit 471 does not force attention with a hard mask, but induces attention only with _{specific information (ie, m ij =1) among incomplete atomic mapping information to cross} It can cause the attention matrix 911 to gradually learn the complete atomic mapping.

The decoder cross-attention unit 472 may output a cross-attention matrix 911 to which masking is applied.

The processor 110 may determine elements to be focused on when calculating an attention loss from the cross-attention matrix 911 to which the masking is applied. The processor 110 may obtain an attention loss of the cross attention matrix 911 based on elements to be attended to. The attention loss may mean an error between the second mask matrix 482 and the cross attention matrix 911 . In an embodiment, the attention loss may be determined by the following Equation (6).

In Equation 6, L _attn may mean an attention loss, M _cross may mean the second mask matrix 482 , and A _cross may mean a cross attention matrix 911 . In addition, ⊙ may mean a Hadamard product.

The processor 110 may obtain a cross entropy loss from the second output sequence. In an embodiment, the processor 110 may obtain a cross-entropy loss by comparing the second output sequence with the second string information 492 . A method of obtaining the cross-entropy loss is not limited to a specific method.

The processor 110 may calculate a total loss of the inverse synthesis prediction model based on the loss of attention and the loss of cross entropy. The processor 110 may calculate the total loss of the inverse synthesis prediction model using Equation 7 below.

는 전체 손실과 어텐션 손실의 균형을 위한 조정 가능한 매개 변수(parameter)일 수 있다. 예를 들어, 매개 변수는 1로 설정될 수 있으나 이에 제한되지 않는다.In Equation 7, L _total means the total loss of the _{retrosynthesis} prediction model, L attn means the attention loss,

may be an adjustable parameter for balancing the total loss and the attention loss. For example, the parameter may be set to 1, but is not limited thereto.

Since the first mask matrix 442 contributes to the cross entropy loss through the output of the desynthesis prediction model, the masking effect of the first mask matrix 442 may be reflected in the cross entropy loss.

The processor 110 may train the inverse synthesis prediction model so that the overall loss of the inverse synthesis prediction model is small.

As the processor 110 removes unnecessary elements and calculates the losses of the inverse synthesis prediction model by paying attention to specific elements, efficiency, speed, and accuracy of learning may be improved.

12 is a flowchart illustrating a method of operating an inverse synthesis prediction model according to an embodiment.

Referring to FIG. 12 , in step S1210 , the processor 110 determines the first attention information from the first string information 491 of the product based on the first graph information 441 of the product, and based on the determination result The first string information 491 may be encoded.

Since neighboring atoms are chemically related to each other, the first attention information may be information on neighboring atoms adjacent to the encoding target atom for efficient learning of the inverse synthesis prediction model.

Since neighboring atoms are determined by the distance between nodes in the first graph information 441 , the processor 110 performs the first string information 491 based on the first graph information 441 and a preset reference distance. ), it is possible to determine the information to be attended to. In this case, the distance may mean a geodesic distance on the graph. Also, the distance may mean a hop-neighbor distance on the graph.

The processor 110 may determine neighboring nodes in the first graph information 441 , and encode tokens of the first string information 491 based on the determination result.

The processor 110 may output the encoded first string information 491 as a first output sequence.

In step S1220 , the processor 110 determines second attention information from the first graph information 441 and the second graph information 481 of the reactant, and generates second string information 492 of the reactant based on the determination result can be decoded.

The relationship between the product and the reactant may be represented by a cross attention matrix, and the ideal cross attention matrix 911 is an atom-mapping representing the relationship between the atoms included in the product and the atoms included in the reactant. Catch information. Accordingly, in order to efficiently learn the inverse synthesis prediction model, information to be paid attention to in the first graph information 441 and the second graph information 481 may be atomic mapping information. In this case, the atomic mapping information may be set using a flexible maximum common substructure (FMCS) technique that utilizes only a specific pair of 'product atom - reactant atom'.

The processor 110 may determine a specific pair of nodes corresponding to each other in the first graph information 441 and the second graph information 481 , and decode tokens of the second string information 492 based on the determination result. have.

The processor 110 may output the decoded second string information 492 as a second output sequence.

In operation S1230 , the processor 110 may train the inverse synthesis prediction model based on the decoding result of the second string information 492 .

The processor 110 may calculate an attention loss of the cross attention matrix 911 indicating a degree of relevance between tokens of the second character string information 492 and tokens of the first character string information 491 based on the atomic mapping information. Also, the processor 110 may calculate a cross-entropy loss of the inverse synthesis prediction model based on the second output sequence. Also, the processor 110 may calculate the total loss of the inverse synthesis prediction model by summing the loss of attention and the loss of cross entropy.

The processor 110 may train an inverse synthesis prediction model based on the total loss. For example, the processor 110 may train the inverse synthesis prediction model until the total loss of the inverse synthesis prediction model becomes smaller than a preset reference loss, but is not limited thereto.

13 is a flowchart illustrating an encoding method according to an embodiment.

Referring to FIG. 13 , in step S1310 , the processor 110 may receive first string information 491 and first graph information 441 .

The first character string information 491 may be input to the processor 110 in a SMILES format. The first string information 491 in SMILES format includes atomic tokens (eg, B, C, N, O) and bonds between atoms (eg, -, =, #), parentheses, and whitespace. may contain non-atomic tokens, such as a number of cyclic structures with

The first graph information 441 may be input to the processor 110 in the form of a two-dimensional graph. Graph information may include nodes and edges. A node may contain information about the atoms of a product, and an edge may contain information about the connection relationship of each atom.

In operation S1320 , the processor 110 may generate a self-attention score matrix 511 indicating a degree of relevance between tokens included in the first string information 491 .

The processor 110 may obtain query, key, and value information from the first string information 491 to generate the self-attention score matrix 511 .

The processor 110 may determine the degree of relevance to all keys for each query, and display the determination result in the self-attention score matrix 511 . The self-attention score matrix 511 may include information about a score indicating the relationship between the query and the key. In one embodiment, the self-attention score matrix 511 may be derived by a scaled dot product attention operation of a query and a key.

In operation S1330 , the processor 110 may apply a mask to the self-attention score matrix 511 based on the first graph information 441 .

The processor 110 may generate a first mask matrix 442 based on the first graph information 441 and a preset reference distance in order to mask unnecessary elements in the self-attention score matrix 511 .

The processor 110 may generate a first free mask matrix by setting tokens included in the first string information 491 as rows and columns.

The processor 110 may set any one node among the nodes included in the first graph information 441 as a reference node. Also, the processor 110 may determine neighboring nodes existing at a distance apart from the reference node by a preset reference distance. In this case, the distance may mean a geodesic distance on the graph. Also, the distance may mean a hop neighbor on the graph. The reference distance can be adjusted by setting.

The processor 110 may allocate information about a reference node and neighboring nodes adjacent to the reference node to the first premask matrix. In an embodiment, the processor 110 may allocate “1” to the reference node and neighboring nodes adjacent to each other, and allocate “0” to the remaining nodes. In another embodiment, the processor 110 may allocate "1" to the reference node and neighboring nodes adjacent to the reference node, and allocate "0" to the remaining nodes.

On the other hand, the first character string information 491 is, in addition to the atomic tokens, bonds between atoms (eg, -, =, #), parentheses, such as the number of cyclic structures having a space (whitespace). Since non-atomic tokens are further included, the tokens of the first string information 491 and the nodes of the first graph information 441 do not match each other. Since these non-atomic tokens can be disambiguated from the full context of string information, a wider range of information may be required. Accordingly, the processor 110 may not mask the nodes corresponding to the non-atomic tokens. In other words, the processor 110 may allocate “1” to nodes corresponding to the non-atomic tokens.

Processor 110 reverse synthesis In order to enhance the learning of the predictive model, different reference distances may be set for each head of the encoder unit 430 . For example, the processor 110 may set the reference distance of the first head as the first distance, and set a second distance different from the first distance to a second head different from the first head.

Based on the first mask matrix 442 , the processor 110 determines elements to be attended to when encoding the first string information 491 in the self-attention score matrix 511 , and based on the determination result, the self-attention score matrix 511 determines the self-attention score matrix 511 . An attention score matrix 511 may be output.

The processor 110 may specify elements having a value of “1” among elements of the first mask matrix 442 . Also, the processor 110 may determine the elements of the self-attention score matrix 511 that correspond to the specified elements (ie, the coordinates are the same). In addition, the processor 110 does not change the values of the elements corresponding to the elements specified in the first mask matrix 442 among the elements of the self-attention score matrix 511, and changes the remaining elements to "-∞". can Accordingly, the processor 110 may determine elements to be attended to among elements of the self-attention score matrix 511 .

In step S1340 , the processor 110 may generate a self-attention matrix indicating the degree of relevance between tokens included in the first string information 491 as a probability based on the self-attention score matrix 511 to which the mask is applied. .

The processor 110 may generate an attention value by calculating the attention distribution of the self-attention score matrix 511 to which the mask is applied using the soft max function, and weighting the calculation result and each value. Attention values may be expressed as a self-attention matrix. The score of the self-attention score matrix 511 may be expressed as a probability by the soft max function.

In operation S1350 , the processor 110 may output the encoded first output sequence based on the self-attention matrix.

14 is a flowchart illustrating a decoding method according to an embodiment.

Referring to FIG. 14 , in step S1410 , the processor 110 may receive second string information 492 and second graph information 481 .

The second character string information 492 may be input to the processor 110 in a SMILES format. The second string information 492 in SMILES format includes an atomic token (eg, B, C, N, O) and bonds between atoms (eg, -, =, #), parentheses, and whitespace. may contain non-atomic tokens, such as a number of cyclic structures with

The second graph information 481 may be input to the processor 110 in the form of a two-dimensional graph. Graph information may include nodes and edges. A node may contain information about the atoms of a product, and an edge may contain information about the connection relationship of each atom.

In step S1420 , the processor 110 generates a cross-attention matrix 911 representing the degree of correlation between tokens included in the first character string information 491 and tokens included in the second character string information 492 as a probability. can do.

The processor 110 may obtain key and value information from the first string information 491 to generate the cross attention matrix 911 . Also, the processor 110 may obtain query information from the second string information 492 .

The processor 110 may determine the degree of relevance to all keys for each query, and display the determination result in a cross-attention score matrix. The cross-attention score matrix may include information about a score indicating a relationship between a query and a key. In one embodiment, the cross attention score matrix may be derived by a scaled dot product attention operation of a query and a key.

The processor 110 may generate an attention value by calculating the attention distribution of the cross attention score matrix using the soft max function, and weighting the calculation result and each value. Attention values may be expressed as a cross attention matrix 911 .

In operation S1430 , the processor 110 may apply a masking to the cross attention matrix 911 based on atomic mapping information indicating a relationship between the atoms included in the product and the atoms included in the reactant.

The processor 110 may obtain atomic mapping information indicating a relationship between an atom included in a product and an atom included in a reactant based on the first graph information 441 and the second graph information 481 .

The processor 110 may acquire atomic mapping information using a flexible maximum common substructure (FMCS) technique. For example, a flexible maximum common substructure (FMCS) may be, but is not limited to, an FMCS algorithm implemented in RDkit.

The processor 110 may determine whether each of the atoms included in the product and the atoms included in the reactant correspond to each other based on the atomic mapping information, and generate the second mask matrix 482 based on the determination result.

The processor 110 may set the second string information 492 as a row and set the first string information 491 as a column to generate a second free mask matrix.

The processor 110 may set any one node among the nodes included in the first graph information 441 as a reference node. Also, the processor 110 may determine a node corresponding to the reference node from among the nodes included in the second graph information 481 .

The processor 110 may allocate information on a node corresponding to the reference node to the second premask matrix. In an embodiment, the processor 110 may allocate “1” to a node corresponding to a reference node and allocate “0” to the remaining nodes.

In one embodiment, the processor 110 may not mask nodes corresponding to non-atomic tokens. In other words, the processor 110 may allocate “1” to nodes corresponding to the non-atomic tokens. In another embodiment, the processor 110 may mask nodes corresponding to non-atomic tokens in order to attend to correspondences between atoms. In other words, the processor 110 may allocate “0” to nodes corresponding to the non-atomic tokens.

The processor 110 determines elements to be attended when calculating the attention loss of the desynthesis prediction model in the cross-attention matrix 911 based on the second mask matrix 482 , and based on the determination result, the cross-attention matrix to which the mask is applied (911) can be output.

The processor 110 may specify elements having a value of “1” among the elements of the second mask matrix 482 . Also, the processor 110 may determine that elements of the cross-attention matrix 911 corresponding to the specified elements (ie, the coordinates are the same) are elements to be attended to when calculating the cross-attention loss of the inverse synthesis prediction model.

In operation S1440 , the processor 110 may output the decoded second output sequence based on the cross-attention matrix 911 to which the mask is applied.

15 is a flowchart illustrating a method of learning an inverse synthesis prediction model according to an embodiment.

Referring to FIG. 15 , in step S1510 , the processor 110 may obtain the attention loss of the desynthesis prediction model from the cross-attention matrix 911 to which the mask is applied. The attention loss may mean an error between the second mask matrix 482 and the cross attention matrix 911 .

In step S1520 , the processor 110 may obtain a cross entropy loss from the second output sequence. In an embodiment, the processor 110 may obtain a cross-entropy loss by comparing the second output sequence with the second string information 492 . A method of obtaining the cross-entropy loss is not limited to a specific method.

In step S1530 , the processor 110 may train the inverse synthesis prediction model based on the loss of attention and the loss of cross entropy.

The processor 110 may calculate the total loss of the inverse synthesis prediction model by summing the loss of attention and the loss of cross entropy.

The processor 110 may train the inverse synthesis prediction model so that the overall loss of the inverse synthesis prediction model is small. For example, the processor 110 may train the inverse synthesis prediction model until the total loss of the inverse synthesis prediction model becomes smaller than a preset reference loss, but is not limited thereto.

While the reverse synthesis prediction method using a template requires domain knowledge of an experienced chemist, the neural network apparatus 100 of the present disclosure performs reverse synthesis prediction without a template, thereby increasing time and cost effectiveness. In addition. The neural network device 100 of the present disclosure may predict reactant combinations corresponding to products beyond the template coverage.

The reverse synthesis prediction method using only the string information of the compound has poor accuracy, and the reverse synthesis prediction method using only the graph information of the compound relies too much on atomic mapping information, whereas the neural network device of the present disclosure is the “string information-graph information” of the present disclosure. By using duality, the speed, accuracy, and efficiency of the inverse synthesis prediction model can be increased.

Meanwhile, the above-described embodiments can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium. In addition, the structure of data used in the above-described embodiments may be recorded in a computer-readable recording medium through various means. The computer-readable recording medium includes a storage medium such as a magnetic storage medium (eg, ROM, floppy disk, hard disk, etc.) and an optically readable medium (eg, CD-ROM, DVD, etc.).

Those of ordinary skill in the art related to the present embodiment will understand that the embodiment may be implemented in a modified form without departing from the essential characteristics of the above description. Therefore, the disclosed embodiments are to be considered in an illustrative rather than a restrictive sense. The scope of the rights is indicated in the claims rather than the above description, and all differences within the scope equivalent thereto should be construed as being included in the present embodiment.

100: neural network device
110: processor
430: encoder unit
440: first mask matrix generator
470: decoder unit
480: second mask matrix generator

Claims (15)

In the learning method of the retrosynthesis prediction model,
determining first attention information from the first character string information of the product based on the first graph information of the product, and encoding the first character string information based on the determination result;
determining second attention information from the first graph information and second graph information of the reactant, and decoding second string information of the reactant based on the determination result; and
and learning the inverse synthesis prediction model based on the decoding result of the second character string information. According to claim 1,
The step of encoding the first string information is
receiving the first character string information and the first graph information;
generating a self-attention score matrix indicating a degree of relevance between tokens included in the first string information;
applying a mask to the self-attention score matrix based on the first graph information;
generating a self-attention matrix representing the degree of attention of each of the tokens included in the first string information as a probability based on the self-attention score matrix to which the mask is applied; and
outputting an encoded first output sequence based on the self-attention matrix. 3. The method of claim 2,
The step of generating the self-attention matrix is
obtaining a query, a key, and a value from the first character string information; and
generating the self-attention matrix based on the query, key, and value. 3. The method of claim 2,
Applying the mask
generating a first mask matrix based on the first graph information and a preset reference distance; and
Based on the first mask matrix, determining elements to be attended to when encoding the first string information in the self-attention score matrix, and outputting a self-attention score matrix to which a mask is applied based on the determination result A method comprising; 5. The method of claim 4,
The step of generating the first mask matrix includes:
setting any one node among the nodes included in the first graph information as a reference node; and
and expressing the reference node and adjacent nodes existing at a distance by the reference distance from the reference node as “1” and expressing the remaining nodes as “0”. According to claim 1,
The step of decoding the second string information is
receiving the second character string information and the second graph information;
generating a cross attention matrix indicating a degree of relevance between tokens included in the first character string information and tokens included in the second character string information as a probability;
applying a mask to the cross attention matrix based on atom mapping indicating a relationship between atoms included in the product and atoms included in the reactant; and
outputting a decoded second output sequence based on the cross-attention matrix to which the mask is applied. 7. The method of claim 6,
The step of generating the cross attention matrix is
obtaining a key and a value from the first character string information;
obtaining a query from the second character string information; and
generating the cross attention matrix based on the query, key and value. 7. The method of claim 6,
Applying the mask
obtaining the atomic mapping information based on the first graph information and the second graph information;
determining whether each of the atoms included in the product and the atoms included in the reactant correspond to each other based on the atomic mapping information, and generating a second mask matrix based on the determination result; and
Based on the second mask matrix, determining elements to be attended to when calculating an attention loss of the inverse synthesis prediction model in the cross-attention matrix, and outputting a cross-attention matrix to which a mask is applied based on the determination result How to include ;. 9. The method of claim 8,
The step of generating the second mask matrix is
setting any one node among the nodes included in the first graph information as a reference node; and
and expressing a node corresponding to the reference node as “1” among the nodes included in the second graph information and expressing the remaining nodes as “0”. 9. The method of claim 8,
The step of training the inverse synthesis prediction model is
obtaining an attention loss of the inverse synthesis prediction model from the cross-attention matrix to which the mask is applied;
obtaining a cross entropy loss of the inverse synthesis prediction model from the second output sequence; and
Training the inverse synthesis prediction model based on the loss of attention and the loss of cross entropy. 11. The method of claim 10,
The method in which the attention loss is tunable by parameters. According to claim 1,
The first character string information and the second character string information are
A method in the form of a Simplified Molecular-Input Line-Entry System (SMILES) code. According to claim 1,
The first graph information and the second graph information
at least one node and at least one edge;
said node contains information about atoms of said product or said reactant;
The edge includes information about the connection relationship of the atoms. A computer-readable recording medium in which a program for executing the method of claim 1 in a computer is recorded. In the apparatus for predicting a reaction product using a reverse synthesis prediction model,
a memory in which at least one program is stored; and
a processor executing the at least one program;
the processor is
determine first attention information from the first character string information of the product based on the first graph information of the product, and encode the first character string information based on the determination result;
determining second attention information from the first graph information and second graph information of the reactant, and decoding second string information of the reactant based on the determination result;
An apparatus for learning the inverse synthesis prediction model based on the decoding result of the second character string information.

Images