JP2024064115A - Image-to-text conversion system - Google Patents

Abstract

[Problem] To convert an image into a character string. [Solution] The image-character string conversion system of the present invention comprises a receiving means for receiving an image, a derivation means for deriving a character string corresponding to the image, and an output means for outputting the derived character string. The image-character string conversion system may form a part of a chemical structure estimation system. The chemical structure estimation system comprises a first generating means for generating an image by converting a chemical structure, a second generating means for modifying the generated image to generate an altered image, a derivation means for deriving a character string corresponding to the altered image, and an output means for outputting a chemical structure corresponding to the derived character string. [Selected Figure] Figure 1C

Description

The present invention relates to a system, method, and program for converting an image into a character string, and to applications of the same, and more specifically, to a system, method, and program for converting an image representing a chemical structure into a character string representing the chemical structure, and to applications of the same. It also relates to a system, method, and program for estimating a chemical structure with modified properties, and to applications of the same.

A technology has been developed that converts chemical structures into unique images for handling (Non-Patent Document 1).

Atsushi Yoshimori, "Prediction of Molecular Properties Using Molecular Topographic Map", Molecules 2021, 26, 4475

In light of the fact that the technology for converting chemical structures into unique images and handling them is an irreversible conversion, the inventors of the present invention have developed a new method for converting an image back into a unique chemical structure. This method converts an image into a character string.

The present invention provides, for example, the following items.
(Item 1)
A receiving means for receiving an image;
A derivation means for deriving a character string corresponding to the image;
and an output means for outputting the derived string.
(Item 2)
The system described in the above item, wherein the derivation means, when inputting a training image and a part of a character of a string corresponding to the training image, derives a character string corresponding to the image using a trained model that has been trained to output the next character after the character.
(Item 3)
The trained model is
outputting a probability of each of a plurality of candidate characters being the next character;
and outputting the next character based on the probability.
(Item 4)
outputting the next character based on the probability
selecting one candidate character from the plurality of candidate characters that has the highest probability; and
and outputting the selected candidate character as the next character.
(Item 5)
outputting the next character based on the probability
selecting a candidate character from the plurality of candidate characters according to the probability distribution for each of the plurality of candidate characters;
and outputting the selected candidate character as the next character.
(Item 6)
20. The system of claim 19, further comprising weighting the distribution of probabilities.
(Item 7)
The system of any one of the preceding claims, wherein the image is an image representing a chemical structure.
(Item 8)
The system according to any one of the preceding claims, wherein the character string is a character string representing a chemical structure.
(Item 9)
The system according to any one of the preceding items, wherein the output means converts the derived character string into a structural formula and outputs the structural formula.
(Item 10)
A generating means for converting a character string to generate an image;
A prediction means for predicting a characteristic of the chemical structure by using an image obtained by converting a character string representing the training chemical structure and a second trained model that has learned a relationship between the image and the characteristic of the training chemical structure,
The generating means converts the output character string to generate an image,
The system according to any one of the preceding claims, wherein the prediction means predicts properties of a chemical structure corresponding to the generated image based on the generated image.
(Item 11)
A chemical structure prediction system, comprising:
A first generating means for converting a chemical structure to generate an image;
a second generating means for generating an altered image by altering the generated image;
A derivation means for deriving a character string corresponding to the altered image;
and an output means for outputting a chemical structure corresponding to the derived character string.
(Item 12)
the first generating means converts a first character string representing a first chemical structure to generate a first image, and converts a second character string representing a second chemical structure to generate a second image;
The system according to any one of the preceding claims, wherein the second generation means generates the modified image by combining the first image and the second image.
(Item 13)
A system for predicting a chemical structure having an altered property, comprising:
A first generating means for converting a chemical structure to generate an image;
a second generating means for generating an altered image by altering the generated image;
A derivation means for deriving a plurality of sets of character strings corresponding to the altered image;
a third generating means for generating a plurality of images by transforming each of the derived sets of character strings;
A prediction means for predicting, based on each of the plurality of images, a property of a chemical structure corresponding to each image;
and an identification means for identifying a chemical structure having an altered property based on the predicted property.
(Item 14)
The system of any one of the preceding claims, wherein the property is an activity associated with the chemical structure.
(Item 15)
The system according to any one of the preceding claims, wherein the activity is a biological or chemical activity possessed by a compound corresponding to the chemical structure.
(Item 16)
The system according to any one of the preceding claims, wherein the activity is an enzyme inhibitory activity.
(Item 17)
A system for predicting a structure of an object having an altered characteristic, comprising:
a first generating means for generating an image uniquely associated with the structure of interest;
a second generating means for generating an altered image by altering the generated image;
A derivation means for deriving a plurality of sets of character strings corresponding to the altered image;
a generating means for generating a plurality of images by converting each of the derived sets of character strings;
A prediction means for predicting characteristics corresponding to each of the plurality of images based on each of the images;
and an identification means for identifying a structure of the object having an altered property based on the predicted property.
(Item 18)
Receiving an image; and
deriving a character string corresponding to the image; and
and outputting the derived string.
(Item 18A)
19. The method of claim 18, comprising the features according to one or more of the preceding claims.
(Item 19)
A program, the program being executed in a computer having a processor unit, the program comprising:
Receiving an image; and
deriving a character string corresponding to the image; and
and outputting the derived character string.
(Item 19A)
20. The program according to claim 19, comprising one or more of the features according to the preceding claims.
(Item 19B)
A computer-readable storage medium storing the program according to item 19 or 19A.
(Item 20)
A chemical structure prediction method, comprising the steps of:
Transforming the chemical structure to generate an image;
modifying the generated image to generate an altered image;
deriving a character string corresponding to the altered image;
and outputting a chemical structure corresponding to the derived string.
(Item 18A)
21. The method of claim 20, comprising the features according to one or more of the preceding claims.
(Item 21)
A chemical structure estimation program, the program being executed on a computer having a processor unit, the program comprising:
Transforming the chemical structure to generate an image;
modifying the generated image to generate an altered image;
deriving a character string corresponding to the altered image;
and outputting a chemical structure corresponding to the derived character string.
(Item 21A)
22. The program according to claim 21, comprising one or more of the features according to the preceding claims.
(Item 21B)
A computer-readable storage medium storing the program according to item 21 or 21A.
(Item 22)
A method for predicting a chemical structure with modified properties, comprising:
Transforming the chemical structure to generate an image;
modifying the generated image to generate an altered image;
deriving a plurality of sets of character strings corresponding to the altered image;
transforming each of the derived sets of strings to generate a plurality of images;
predicting, based on each image of the plurality of images, a property of a chemical structure corresponding to the respective image;
and identifying a chemical structure having an altered property based on the predicted property.
(Item 22A)
23. The method of claim 22, comprising the features according to one or more of the preceding claims.
(Item 23)
A program for predicting a chemical structure having modified properties, the program being executed on a computer having a processor unit, the program comprising:
Transforming the chemical structure to generate an image;
modifying the generated image to generate an altered image;
deriving a plurality of sets of character strings corresponding to the altered image;
transforming each of the derived sets of strings to generate a plurality of images;
predicting, based on each image of the plurality of images, a property of a chemical structure corresponding to the respective image;
and identifying a chemical structure having an altered property based on the predicted property.
(Item 23A)
24. The program according to claim 23, comprising one or more of the features according to the preceding claims.
(Item 23B)
A computer-readable storage medium storing the program according to item 23 or 23A.
(Item 24)
A method for predicting the structure of an object having an altered property, comprising:
generating an image uniquely associated with the structure of interest;
modifying the generated image to generate an altered image;
deriving a plurality of sets of character strings corresponding to the altered image;
transforming each of the derived sets of strings to generate a plurality of images;
predicting, based on each image of the plurality of images, a characteristic corresponding to the respective image;
and identifying a structure of the subject having an altered property based on the predicted property.
(Item 24A)
25. The method of claim 24, comprising the features according to one or more of the preceding claims.
(Item 25)
A program for predicting a structure of an object whose characteristics have been modified, the program being executed on a computer having a processor unit, the program comprising:
generating an image uniquely associated with the structure of interest;
modifying the generated image to generate an altered image;
deriving a plurality of sets of character strings corresponding to the altered image;
transforming each of the derived sets of strings to generate a plurality of images;
predicting, based on each image of the plurality of images, a characteristic corresponding to the respective image;
and identifying a structure of the target having an altered property based on the predicted property.
(Item 25A)
26. The program according to claim 25, comprising one or more of the features according to the preceding claims.
(Item 25B)
A computer-readable storage medium storing the program according to item 25 or 25A.

According to the present invention, it is possible to provide a system for converting an image into a character string. This makes it possible to convert, for example, an image into a chemical structure represented by a character string. In addition, this system can be used to estimate a chemical structure (for example, a novel chemical structure, or a chemical structure having modified properties).

FIG. 1 is a diagram showing an example of a flow of chemical structure estimation by the chemical structure estimation system 1000. FIG. 1 is a diagram showing another example of a flow for predicting a chemical structure by the chemical structure prediction system 1000. A diagram showing the concept of the image-to-character string conversion system 100 (MTM inverter) converting MTM back into a chemical structure. A diagram showing the concept of how the image-to-character string conversion system 100 converts an image into a corresponding character string. A diagram showing the concept of the image-to-character string conversion system 100 converting an image showing the physical properties of a compound into a corresponding character string. FIG. 1 shows an example of the configuration of an image-character string conversion system 100. FIG. 1 is a diagram showing an example of a specific configuration of an image-character string conversion system 100. FIG. 1 shows an example of the configuration of a processor unit 120. FIG. 1 shows an example of the configuration of a processor unit 120A that is an alternative embodiment of the processor unit 120. FIG. 1 shows an example of the configuration of a processor unit 120B that is an alternative embodiment of the processor unit 120. FIG. 1 shows an example of the configuration of a processor unit 120C that is an alternative embodiment of the processor unit 120. A diagram showing an example of the structure of a trained model A diagram showing an example of the structure of a trained model A flowchart showing an example of processing (process 600) in the image-character string conversion system 100. A flowchart showing an example (700) of a process in the chemical structure estimation system 1000. A flowchart showing an example (800) of a process in the chemical structure estimation system 1000. FIG. 1 is a diagram showing distributions of similarities in training data and test data in an embodiment; FIG. 1 shows an example of a compound with the best predictive value in an embodiment. A diagram showing four types classified in the embodiment. A diagram showing four types classified in the embodiment.

The present disclosure will be described below. Throughout this specification, singular expressions should be understood to include the concept of the plural form, unless otherwise specified. Therefore, singular articles (e.g., in the case of English, "a", "an", "the", etc.) should be understood to include the concept of the plural form, unless otherwise specified. In addition, it should be understood that the terms used in this specification are used in the sense commonly used in the field, unless otherwise specified. Therefore, unless otherwise defined, all technical terms and scientific and technical terms used in this specification have the same meaning as commonly understood by those skilled in the art to which this invention belongs. In the case of conflict, the present specification (including definitions) will take precedence.

(Definition)
In this specification, a "character string" is a character string that can represent the structure of an object. When the object is a substance, the structure of the object is a chemical structure. The character string that can represent a chemical structure may be, for example, a character string in the SMILES notation.

In this specification, a "character string corresponding to an image" refers to a character string that has some relationship with the target image. The relationship may be some mathematical relationship, or it may be any other relationship such as related to chemistry. This character string may have a one-to-one correspondence with the image, or may be one of a group of character strings that have a similar relationship to the character string that has a one-to-one correspondence with the image. This character string may be, for example, a character string derived by a probability distribution.

As used herein, "chemical structure" refers to a structure showing the constituent atoms of a substance (e.g., a compound) including the state of bonding. A "chemical structure" can be represented in various ways. For example, a chemical structure can be represented by a chemical formula, more specifically, a structural formula. For example, a chemical structure can be represented by a character string. Representing a chemical structure by a character string is known, for example, as the SMILES notation. For example, a chemical structure can be represented by an image. Representing a chemical structure using an image can be achieved by converting the chemical structure into a Molecular Topographic Map (MTM) invented by the present inventor (e.g., Atsushi Yoshimori, "Prediction of Molecular Properties Using Molecular Topographic Map", Molecules 2021, 26, 4475).

As used herein, "characteristics of a chemical structure" refers to properties possessed by a substance having that chemical structure, and includes any property. Examples of properties include chemical properties, physical properties, and mechanical properties.

In this specification, "structural formula" refers to a formula that diagrammatically represents the bonding state of atoms within a molecule.

As used herein, "modifying" an image refers to changing the pixel value of at least one pixel of the image if the image is two-dimensional, or to changing the voxel value of at least one voxel of the image if the image is three-dimensional. The pixel or voxel value is changed, for example, by adding, subtracting, multiplying, or dividing a given value. The given value can be the pixel value of a corresponding pixel or voxel value of a corresponding voxel of another image. Modifying an image with the pixel or voxel values of another image is called compositing the image with the other image.

As used herein, "activity" refers to activity associated with a chemical structure, and preferably refers to biological or chemical activity possessed by a compound corresponding to the chemical structure. Activity can be expressed, for example, by a pKi value > 6 or a pIC50 value > 6, preferably a pKi value > 8 or a pIC50 value > 8.

In this specification, "about" means ±10% of the numerical value that follows.

Preferred Embodiments
Preferred embodiments of the present disclosure are described below. The embodiments provided below are provided for a better understanding of the present disclosure, and it is understood that the scope of the present disclosure should not be limited to the following description. Therefore, it is clear that a person skilled in the art can make appropriate modifications within the scope of the present disclosure in light of the description in this specification. It is also understood that the following embodiments can be used alone or in combination.

The following describes a preferred embodiment of the present disclosure with reference to the drawings.

1. Chemical Structure Prediction System The inventors of the present disclosure have developed a new system for predicting chemical structures. This chemical structure prediction system can search for new chemical structures. For example, when the chemical structure of one known compound is input to the chemical structure prediction system, the system outputs the chemical structure of a new compound that may be similar to the known compound. For example, when multiple known compounds are input to the chemical structure prediction system, the system can output a new compound that may be the average of the multiple compounds.

Figure 1A shows an example of a flow for estimating a chemical structure using the chemical structure estimation system 1000. In this example, we will explain an example of estimating a compound that is the average of two compounds based on their chemical structures.

In step S1, the chemical structures of two compounds (a first compound C1 and a second compound C2) are input to the chemical structure estimation system 1000. The chemical structures of the two compounds (C1, C2) are input, for example, as structural formulas.

When the chemical structures of two compounds (C1, C2) are input to the chemical structure estimation system 1000, the chemical structure estimation system 1000 generates an image (e.g., MTM) representing the chemical structure of each compound. A chemical structure image I1 of the first compound C1 is generated, and an image I2 is generated from the chemical structure of the second compound C2.

The MTM may be, for example, a heat map of 28 x 28 pixels. The heat map is a diagram in which the value corresponding to each pixel is represented by a color or shade, and may be a color map, a grayscale map, or a map represented in another format. For example, as shown in the MTM of FIG. 1C described below, the color or shade may be associated with the value of a predetermined parameter. A unique MTM is generated from the chemical structure of each compound. That is, the chemical structure of the first compound C1 and MTM I1 correspond one-to-one, and the chemical structure of the second compound C2 and MTM I2 correspond one-to-one.

In the chemical structure estimation system 1000, by treating the chemical structure as an image (e.g., MTM), it is possible to perform calculations (e.g., addition, subtraction) on the chemical structure that would be difficult to perform on the structural formula.

In step S2, image I1 generated from the chemical structure of the first compound C1 and image I2 generated from the chemical structure of the second compound C2 are synthesized. Here, image I1 generated from the chemical structure of the first compound C1 and image I2 generated from the chemical structure of the second compound C2 are synthesized to take the average. For example, in the synthesized image I3, the pixel value of each pixel can be the average of the pixel values of the corresponding pixels of the two images I1 and I2. The average may be a weighted average. In this case, the weight can be set to an arbitrary value.

The synthesized image I3 is an image of the chemical structure of a new compound, neither the chemical structure of the first compound C1 nor the chemical structure of the second chemical structure C2. As it is, it is impossible to understand what kind of chemical structure this new compound's chemical structure image I3 has. Therefore, the chemical structure estimation system 1000 converts this new compound's chemical structure image I3 into a character string representing the chemical structure of that compound.

In step S3, image I3 is converted into a character string representing the compound. The character string representing the compound may be, for example, a character string expressed according to the SMILES notation. This makes it possible to understand what kind of chemical structure image I3 represents. For example, when the character string expressed according to the SMILES notation is expressed as a structural formula, it is expressed as a new compound C3.

The output new compound C3 may be an average compound of the first compound C1 and the second compound C2, and is expected to have properties between those of the first compound C1 and those of the second compound C2, or properties that are improved over those of the first compound C1 and those of the second compound C2.

In this way, the chemical structure estimation system 1000 can estimate a new compound that has properties similar to those of the first compound C1 and the second compound C2 or that is superior to those of the first compound C1 and the second compound C2. The new compounds estimated in this way are expected to be applied in a variety of drug discovery applications.

In the above example, the chemical structures of two compounds are input to the chemical structure estimation system 1000. However, for example, the chemical structures of three or more compounds may be input to the chemical structure estimation system 1000, or the chemical structure of one compound may be input to the chemical structure estimation system 1000. When the chemical structures of three or more compounds are input to the chemical structure estimation system 1000, the chemical structure estimation system 1000, for example, averages images generated from the chemical structures of the three or more compounds, and estimates the chemical structure of the compound that is the average of the three or more compounds. When the chemical structure of one compound is input to the chemical structure estimation system 1000, the chemical structure estimation system 1000, for example, modifies an image generated from the chemical structure of one compound, and estimates a chemical structure similar to the one compound.

In the above example, the chemical structure estimation system 1000 has been described as converting image I3 uniquely into a character string. In this case, when the character string obtained by converting image I3 uniquely into an image is converted again, image I3 and the reconverted image may be similar but not identical. In this case, the chemical structure estimation system 1000 can convert image I3 into multiple character string candidates, thereby deriving a character string (and thus a chemical structure) that matches image I3 or can be converted into an image that is more similar to image I3.

Figure 1B shows another example of a flow for estimating a chemical structure using the chemical structure estimation system 1000. In this example, too, an example is described in which a compound that is the average of two compounds is estimated based on the chemical structures of the two compounds.

In step S11, the chemical structures of two compounds (a first compound C11 and a second compound C12) are input to the chemical structure estimation system 1000. The chemical structures of the two compounds (C11, C12) are input, for example, as structural formulas.

When the chemical structures of two compounds (C11, C12) are input to the chemical structure estimation system 1000, the chemical structure estimation system 1000 generates an image (e.g., MTM) representing the chemical structure of each compound. Image I11 is generated from the chemical structure of the first compound C11, and image I2 is generated from the chemical structure of the second compound C12.

In step S12, image I11 generated from the chemical structure of the first compound C11 and image I12 generated from the chemical structure of the second compound C12 are synthesized. Here, similar to the example of FIG. 1A, image I11 generated from the chemical structure of the first compound C11 and image I12 generated from the chemical structure of the second compound C12 are synthesized to take the average.

The synthesized image I13 is an image of the chemical structure of a new compound, which is neither the chemical structure of the first compound C11 nor the chemical structure of the second compound C12.

In step S13, image I13 is converted into a character string representing the compound. At this time, in the chemical structure estimation system 1000, image I13 is converted into a plurality of character string candidates. Each of the plurality of character string candidates may correspond to a first candidate compound C13', a second candidate compound C13'', a third candidate compound C13''', etc.

The chemical structure estimation system 1000 determines which of the multiple character string candidates (and thus multiple candidate compounds) represents a chemical structure that is appropriate for the chemical structure represented by image I13. For example, by converting each character string candidate into an image and calculating the similarity between each converted image and image I13, it is possible to identify the character string candidate that has been converted into an image with the highest similarity.

In step S14, a character string that is identified as being appropriate for the chemical structure represented by image I13 from among the multiple character string candidates (and thus the multiple candidate compounds) is output. In this example, a character string corresponding to the third candidate compound C13''' is output.

The output new compound C13''' may be an average compound of the first compound C11 and the second compound C12, and is expected to have properties between those of the first compound C11 and those of the second compound C12, or properties that are improved over those of the first compound C11 and those of the second compound C12.

The chemical structure estimation system 1000 can, for example, be used in conjunction with a predictive model that predicts the properties of a chemical structure from an image to estimate a chemical structure with modified (preferably improved) properties. The predictive model is a model trained to learn the relationship between an image (e.g., MTM) converted from a known chemical structure and the known properties of that chemical structure. When an image converted from an unknown chemical structure is input to this predictive model, the predictive model can predict the properties of that chemical structure.

For example, an image converted from each of the multiple character string candidates converted from image I13 in step S13 can be input to the prediction model. This allows the properties of each of the chemical structures corresponding to the multiple character string candidates to be predicted. By comparing the properties of each, the chemical structure estimation system 1000 can output a character string corresponding to the chemical structure having the most improved property. The character string output in this manner represents a chemical structure with modified (preferably improved) properties.

For example, if compounds with strong biological activity (e.g., enzyme inhibitory activity) are selected as the first compound C11 and the second compound C12, the chemical structure estimation system 1000 can estimate a chemical structure with improved biological activity.

The above-mentioned chemical structure estimation system 1000 includes an image-to-character string conversion system 100. The image-to-character string conversion system 100 can convert an input image into a character string representing the image. Specifically, when an image representing a chemical structure (e.g., an MTM) is input to the image-to-character string conversion system 100, a character string corresponding to the image (e.g., the SMILES of the chemical structure represented by the image) can be output. The image-to-character string conversion system 100 can perform the above-mentioned steps S3 and S13.

In this way, the image-to-character string conversion system 100 can also be called an MTM inverter, since it converts the MTM converted from a chemical structure back into a character string representing the chemical structure.

Figure 1C shows the concept of the image-to-character string conversion system 100 (MTM inverter) converting MTM back into a chemical structure.

The MTM inverter 100 can receive an MTM as input. The MTM inverter converts the MTM to a SMILES using a model trained to learn the relationship between the MTM and a string of characters (SMILES) that represent the chemical structure that the MTM represents.

For example, when the MTM shown in Figure 1C is input to an MTM inverter, the SMILES "Cc1nnc2n1-c1sc(C#CC3CCOCC3)c(Cc3ccccc3)c1COC2" is output. This SMILES represents the compound 3-benzyl-9-methyl-2-((tetrahydro-2H-pyran-4-yl)ethyl)-4H,6H-thieno[2,3-e][1,2,4]triazolo[3,4-c][1,4]oxazepine.

In the above example, it was explained that when an image representing a chemical structure is input to the image-to-character string conversion system 100 (MTM inverter 100), a character string representing the chemical structure corresponding to the image is output, but the present invention is not limited to this. The image-to-character string conversion system 100 can be applied to any image as long as the image and the character string can be associated.

Figure 1D shows the concept of how the image-to-string conversion system 100 converts an image into a corresponding string.

An image is input to the image-to-character string conversion system 100. The image may be an image representing a chemical structure as described above, an image representing a gene sequence, or an image showing the physical properties of a compound. The image-to-character string conversion system 100 converts the image into the corresponding character string using a model that has been trained to learn the relationship between the image and the character string corresponding to the image.

For example, when an image showing a gene sequence is input to the image-character string conversion system 100, a character string corresponding to the gene sequence shown by the image can be output. For example, as shown in FIG. 1E, when an image showing an image showing the physical properties of a compound is input to the image-character string conversion system 100, a character string showing the chemical structure of the compound shown by the image can be output. The image showing the physical properties of a compound is an image having multiple squares, each of which represents a physical property of the compound. The image showing the physical properties of a compound can be, for example, a heat map of 3×3 squares. For example, as shown in FIG. 1E, each square can represent the molecular weight, solubility, hERG inhibitory activity, membrane permeability, BBB permeability, CYP1A2 inhibitory activity, CYP2C19 inhibitory activity, CYP2C9 inhibitory activity, and CYP2D6 inhibitory activity by color or shade.

The image-to-character string conversion system 100 described above may have, for example, the configuration described below.

2. Configuration of image-to-character string conversion system 100

Figure 2 shows an example of the configuration of an image-to-character string conversion system 100.

The system 100 is connected to a database unit 200. The system 100 is also connected to at least one terminal device 300 via a network 400.

Note that although three terminal devices 300 are shown in FIG. 2, the number of terminal devices 300 is not limited to this. Any number of terminal devices 300 may be connected to the system 100 via the network 400.

Network 400 may be any type of network. Network 400 may be, for example, the Internet or a LAN. Network 400 may be a wired network or a wireless network.

An example of the system 100 may be, for example, a computer (e.g., a server device) installed in a provider P that provides a chemical structure estimation service, or a computer (e.g., a server device) installed in a provider P that provides a web application for chemical structure estimation, but is not limited thereto. The system 100 may be, for example, a computer (e.g., a terminal device) installed in a user (e.g., a pharmaceutical company), and an application for chemical structure estimation is installed in the user's computer. In this case, the system 100 does not need to be connected to the network 400. The terminal device may be any type of terminal device, such as a smartphone, tablet, personal computer, smart glasses, or smart watch.

The database unit 200 may store character strings output by the image-character string conversion system 100 or images converted from character strings. Alternatively, the database unit 200 may store character strings or images representing chemical structures output by the chemical structure estimation system 1000. Alternatively, the database unit 200 may store training data for constructing a trained model or a constructed trained model.

Figure 3 shows an example of a specific configuration of the image-to-character string conversion system 100.

The system 100 includes an interface unit 110, a processor unit 120, and a memory unit 130.

The interface unit 110 exchanges information with the outside of the system 100. The processor unit 120 of the system 100 can receive information from the outside of the system 100 via the interface unit 110, and can transmit information to the outside of the system 100. The interface unit 110 can exchange information in any format.

The interface unit 110 includes, for example, an input unit that allows information to be input to the system 100. It does not matter in what manner the input unit allows information to be input to the system 100. For example, if the input unit is a touch panel, the user may input information by touching the touch panel. Alternatively, if the input unit is a mouse, the user may input information by operating the mouse. Alternatively, if the input unit is a keyboard, the user may input information by pressing a key on the keyboard. Alternatively, if the input unit is a microphone, the user may input information by inputting voice into the microphone. Alternatively, if the input unit is a data reading device, the information may be input by reading information from a storage medium connected to the system 100. Alternatively, if the input unit is a receiver, the receiver may input information from outside the system 100 via a network. In this case, the type of network does not matter. For example, the receiver may receive information via the Internet or via a LAN.

The interface unit 110 includes, for example, an output unit that enables information to be output from the system 100. It does not matter in what manner the output unit enables information to be output from the system 100. For example, if the output unit is a display screen, it may be configured to output information on the display screen. Alternatively, if the output unit is a data writing device, it may be configured to output information by writing information to a storage medium connected to the system 100. Alternatively, if the output unit is a printing device, it may be configured to output information by printing on a medium such as paper. Alternatively, if the output unit is a transmitter, it may be configured to output information by transmitting the information to the outside of the system 100 via a network. In this case, the type of network does not matter. For example, the transmitter may transmit information via the Internet or via a LAN.

The system 100 can transmit information to the database unit 200 and/or receive information from the database unit 200, for example, via the interface unit 110. The system 100 can transmit information to the terminal device 300 and/or receive information from the terminal device 300, for example, via the interface unit 110.

The system 100 can receive an image representing a chemical structure, for example, via the interface unit 110.

The system 100 can output, for example, a character string representing a compound via the interface unit 110. The system 100 can output, for example, a structural formula converted from the character string representing a compound via the interface unit 110.

The processor unit 120 executes the processing of the system 100 and controls the operation of the entire system 100. The processor unit 120 reads out a program stored in the memory unit 130 and executes the program. This makes it possible for the system 100 to function as a system that executes desired steps. The processor unit 120 may be implemented by a single processor or by multiple processors.

The memory unit 130 stores programs required to execute the processing of the system 100 and data required to execute the programs. The memory unit 130 may store a program for causing the processor unit 120 to perform a process for converting an image into a character string (for example, a program for realizing the process shown in FIG. 6 described later), a program for causing the processor unit 120B to perform a process for estimating a chemical structure (for example, a program for realizing the process shown in FIG. 7 described later), and/or a program for causing the processor unit 120C to perform a process for estimating a chemical structure whose characteristics have been modified (for example, a program for realizing the process shown in FIG. 8 described later). Here, it does not matter how the program is stored in the memory unit 130. For example, the program may be pre-installed in the memory unit 130. Alternatively, the program may be stored in a computer-readable storage medium and installed in the memory unit 130 by reading the computer-readable storage medium. Alternatively, the program may be installed in the memory unit 130 by being downloaded via a network. In this case, the type of network does not matter. The memory unit 130 may be implemented by any storage means.

The database unit 200 may store, for example, a character string output by the image-character string conversion system 100 or an image converted from a character string. Alternatively, the database unit 200 may store a character string or an image representing a chemical structure output by the chemical structure estimation system 1000. Alternatively, the database unit 200 may store training data for constructing a trained model or a constructed trained model.

3, the database unit 200 is provided outside the system 100, but the present invention is not limited to this. The database unit 200 can also be provided inside the system 100. In this case, the database unit 200 may be implemented by the same storage means as the storage means that implements the memory unit 130, or may be implemented by a storage means different from the storage means that implements the memory unit 130. In any case, the database unit 200 is configured as a storage unit for the system 100. The configuration of the database unit 200 is not limited to a specific hardware configuration. For example, the database unit 200 may be configured as a single hardware component, or may be configured as multiple hardware components. For example, the database unit 200 may be configured as an external hard disk device of the system 100, or may be configured as a cloud storage connected via a network.

Figure 4A shows an example of the configuration of the processor unit 120.

The processor unit 120 includes a receiving means 121, a derivation means 122, and an output means 123.

The receiving means 121 is configured to receive an image.

The image can be any image that can be associated with a string, and is preferably an image representing a chemical structure. The received image is passed to the derivation means 122.

The derivation means 122 is configured to derive a character string corresponding to an image.

The derivation means 122 can derive a character string corresponding to an image, for example, by using a trained model. The trained model is a model that has been trained to output the next character after the input character when a training image and some characters of a character string corresponding to the training image are input.

Figure 5A shows an example of the structure of a trained model. In this example, a deep neural network (DNN) is used. A DNN can have any number of layers.

For example, when an image is input to the first input layer and some characters of a string are input to the second input layer, the next character in the string is output from the output layer 503. This DNN can be constructed, for example, by the following method. Below, an example of an image (MTM) representing a chemical structure and the corresponding string (SMILES) is explained.

First, the chemical structures of multiple known compounds are converted into images, and character strings (SMILES) for each compound are derived and stored as pairs of images and character strings. Character strings can be divided into meaningful words and stored. Words can be, for example, tokens, which are the semantic building blocks of SMILES.

For example, SMILES:
Cn1cc(-c2ccccc2CN2CCOCC2)c2cc[nH]c2c1=O
By token,
start C n 1 c c ( - c 2 c c c c c c 2 C N 2 C C O C C 2 ) c 2 c c [nH] c 2 c 1 = O end
The tokens represented by SMILES start with a start token and end with an end token, with each token separated by a space.

There are a total of 32 types of SMILES tokens, including start, end, and none tokens. Each token is assigned a number between 0 and 31 from the token-number conversion table. The none token is a token that indicates nothing, and is assigned the numerical value 0. The SMILES represented by the tokens can be quantified using the token-number conversion table. The token-number conversion table is constructed from the SMILES contained in the training data, so if the training data changes, the token-number conversion table may also change. An appropriate token-number conversion table may be used for learning.

In one example, the token-number conversion table can be expressed as follows. The tokens in "" correspond to numbers from 0 to 31. Note that the token-number conversion table can vary depending on the number and types of SMILES used in learning.
{"none": 0, "start": 1, "C": 2, "S": 3, "(": 4, "=": 5, "O": 6, ")": 7, "c": 8, "1": 9, "n": 10, "2": 11, "F": 12, "-": 13, "3": 14, "[nH]": 15, "end": 16, "N": 17, "[C@@H]": 18, "4": 19, "L": 20, "[N+]": 21, "[O-]": 22, "s": 23, "o": 24, "[C@H]": 25, "5": 26, "#": 27, "R": 28, "6": 29, "/": 30, "[SH]": 31}

Using this token-to-number conversion table, SMILES represented by the token:
start C n 1 c c ( - c 2 c c c c c c 2 C N 2 C C O C C 2 ) c 2 c c [nH] c 2 c 1 = O end
teeth,
Quantified SMILEs:
[1, 2, 10, 9, 8, 8, 4, 13, 8, 11, 8, 8, 8, 8, 8, 11, 2, 17, 11, 2, 2, 6, 2, 2, 11, 7, 8, 11, 8, 8, 15, 8, 11, 8, 9, 5, 6, 16]
It is expressed as:

As learning data, multiple learning quantified SMILEs are created from one quantified SMILE according to its length.

The training data is a pair of input vectors and output vectors. The pair of input vectors and output vectors can be, for example, a pair of a vector representing the first character of SMILES and a vector representing the second character of SMILES, a pair of a vector representing the first two characters of SMILES and a vector representing the third character of SMILES, a pair of a vector representing the first three characters of SMILES and a vector representing the fourth character of SMILES, etc.

For example, in the above quantified SMILES,
The first learning data is
Input vector:
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1]
Output vector (answer):
[0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
Here, the maximum length of tokenized SMILES is set to 80, and the input vector is an 80-dimensional vector. Empty elements are filled with 0 (indicating a none token). The first character (here, 1 (the "start" token)) is inserted at the end of the input vector.

In addition, since there are 32 types of tokens, the output vector is a 32-dimensional vector. In this example, the second character is 2 (the "C" token), so the second component of the output vector is set to 1.

With this input and output vector pair, the model learns that the input is "start" and the output (answer) is "C."

The second learning data is
Input vector:
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2]
Output vector (answer):
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
The second character is inserted at the end of the input vector, in this case a 2 (the "C" token). The numbers are shifted from the back of the vector to the front. Also, because the third character is 10 (the "n" token), the 10th component of the output vector is set to 1.

With this pair of input and output vectors, the model learns that the input is "start, C" and the output (answer) is "n".

From the above, the input is "start C" and the output (answer) is "n".

The third learning data is
Input vector:
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 10]
Output vector (answer):
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
The third character is inserted at the end of the input vector, in this case a 10 (the "n" token). The numbers are shifted from the back of the vector to the front. Also, because the fourth character is a 9 (the "1" token), the ninth component of the output vector is set to 1.

With this pair of input and output vectors, the model learns that the input is "start, C, n" and the output (answer) is "1."

By repeating this procedure, training data is created until the "end" token is set in the output vector. For example, if the length of the quantified SMILES is 38, 37 pieces of training data are generated. In this example, 37 pieces of training data are generated for one MTM. Training processing is performed using the training data and the MTM. The MTM is, for example, an image of 28 x 28 pixels, and is expressed as a 28 x 28 second-order tensor (two-dimensional matrix). Each component may be a normalized pixel value or a real value.

A trained model is constructed by performing the DNN training process shown in FIG. 5A using the created training data and the corresponding MTM. The hyperparameters used during training can be set to any value.

The derivation means 122 can use the trained model constructed as described above to derive the SMILES corresponding to the MTM as follows:

First, the MTM and an input vector representing the "Start" token are input to the trained model. The MTM is converted from a two-dimensional matrix into a vector consisting of 784 elements, and then input to the first input layer 501 of the trained model. The input vector representing the "Start" token is input to the second input layer 502 of the trained model. The input vector representing the "Start" token may be a vector with a number corresponding to the "Start" token inserted at the end.
Input vector:
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1]

The output layer 503 of the trained model predicts and outputs the token that will come next after the "Start" token. The output of the output layer 503 of the trained model is a 32nd order vector, with each component indicating the probability that the corresponding token will come next.

For example, if the output vector is
[ 7.69443306e-14 3.60606239e-13 9.99999762e-01 4.12664393e-16
9.96908610e-13 2.51735466e-10 9.08047681e-08 1.14084934e-07
6.60184440e-09 3.07496715e-12 2.06361572e-09 3.41530715e-15
1.59075051e-16 1.24587674e-10 1.68514540e-12 4.61158471e-08
1.24160726e-09 2.73644041e-12 7.36772874e-20 1.35796608e-15
2.88039873e-17 1.01721806e-13 1.73131027e-13 6.01821607e-17
1.28117691e-16 6.16416265e-22 1.21871484e-20 1.19068711e-21
1.47954461e-16 3.89847341e-28 4.67776026e-23 5.85218293e-23］
In this case, in one example, the second component has the largest value (probability) of 9.99999762e-01 among the 32 components from the 0th component to the 31st component, so the token corresponding to the second component can be selected (the method of selecting the value with the largest probability is called the Argmax method). Since the second component corresponds to the "C" token according to the token-number conversion table, it is predicted that the next token after the input "start" token will be the "C" token.

In another example, the temperature sampling (T-sampling) method can be used as a selection method different from the Argmax method. The T-sampling method is described, for example, in iScience 25, 104661, 2022 DOI (https://doi.org/10.1016/j.isci.2022.104661). The T-sampling method is a method inspired by statistical thermodynamics, in which the higher the temperature, the more likely it is to encounter a low-energy state. In the Argmax method, only the token with the largest value (probability) is selected, but the T-sampling method is a method of selecting a token based on the temperature T, and is a method that allows the selection of tokens other than the token with the largest value (probability).

The token selection method using the T-sampling method is as follows:

First, the output vector X is transformed to p using equation (1).
Here, z _i is ln(x _i ), where the subscript i refers to the i-th component of the vector, and T is the temperature, which can be set to any value. The larger T is, the smaller the difference in probability of each component becomes, and the more even the probability of the token being selected becomes.

Next, components are selected by random sampling using a multinomial distribution based on p _i . This random sampling has the potential to select components different from those selected by the Argmax method. The use of the T-sampling method makes it possible to generate a variety of character strings.

The selected token is inserted at the end of the input vector. For example, if the token corresponding to the second component is selected by the Argmax method (the "C" token), it is inserted at the end of the input vector, shifting the values from the back of the vector to the front.
Input vector:
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2]

This input vector and MTM are input to the trained model, and a token is selected based on the output probability. This process is repeated, and the process can be terminated when any of the following conditions is met:
1. The "end" token is selected.
2. The input vector has no 0s (none tokens).

The vector obtained in this way is converted to SMILES using a token-to-number conversion table. At this time, the none token, start token, and end token are removed.

for example,
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 8 9 15 8 4 13 8 11 8 8 8 8 8 11 6 8 11 8 8 8 8 8 11 7 8 8 9 2 4 17 7 5 6 16]
When we obtain this, we convert it to SMILES and get
Cc1[nH]c(-c2ccccc2Oc2ccccc2)cc1C(N)=O
This represents the compound 2-methyl-5-(2-phenoxyphenyl)-1H-pyrrole-3-carboxamide.

In this way, a character string can be derived from the MTM. If the T-sampling method is used, multiple character string candidates will be derived.

The output means 123 is configured to output the derived character string.

The output means 123 can, for example, convert the derived character string into information that is easily understandable to the user and output the information. For example, if the derived character string represents a chemical structure, the output means 123 can convert the derived character string into a structural formula and output the structural formula.

The output string or information converted from the string can be presented to the user, for example, via the interface unit 110.

The output string can be used for any purpose. In one example, the output string can be used to predict properties of the corresponding chemical structure.

Figure 4B shows an example of the configuration of processor unit 120A, which is an alternative embodiment of processor unit 120. Processor unit 120A has a configuration for predicting properties of chemical structures.

The processor unit 120A includes a receiving means 121, a derivation means 122, an output means 123, a generation means 124, and a prediction means 125. The receiving means 121, the derivation means 122, and the output means 123 have the same configuration as that described above for the processor unit 120, and a description thereof will be omitted here.

The output means 123 passes the derived string to the generation means 124.

The generating means 124 is configured to convert the character string to generate an image.

Converting the character string to generate an image includes converting the chemical structure corresponding to the character string into an image (MTM). This can be achieved, for example, using the MTM converter described in Atsushi Yoshimori, "Prediction of Molecular Properties Using Molecular Topographic Map", Molecules 2021, 26, 4475.

The prediction means 125 is configured to predict a property of an object based on an image. In particular, the prediction means 125 can predict a property of a chemical structure based on an image representing the chemical structure.

The prediction means 125 can predict the characteristics of a chemical structure, for example, by using a second trained model that has learned the relationship between an image obtained by converting a character string representing a training chemical structure and the characteristics of the training chemical structure. The second trained model has been trained to output the characteristics of the training chemical structure when an image obtained by converting a character string representing a training chemical structure is input.

Figure 5B shows an example of the structure of a trained model. In this example, a deep neural network (DNN) is used. A DNN can have any number of layers.

For example, when an image is input to the input layer, the characteristics of the chemical structure corresponding to the image are output from the output layer. This DNN can be constructed, for example, by using an image generated from one training chemical structure as input training data for each of a number of training chemical structures, and learning the characteristics of that training chemical structure as output training data. The image generated from the training chemical structure is used as the input training data, for example, as a 28 x 28 two-dimensional matrix.

The characteristics predicted by the prediction means 125 can be output to the outside of the system 100. The predicted characteristics may be stored, for example, in the database unit 200, or may be transmitted to the terminal device 300 and presented to the user.

In this way, the processor unit 120A can convert the character string derived from the received image back into an image to predict the properties of the target (e.g., a chemical structure). This is useful, for example, in identifying one character string (or a corresponding chemical structure) when multiple character string candidates are derived from the derivation means 122. Specifically, when multiple character string candidates are derived from the derivation means 122, it becomes possible to select the chemical structure (or corresponding character string) having the best or desired properties by converting each of the character string candidates into an image and predicting the properties.

Figure 4C shows an example of the configuration of processor unit 120B, which is an alternative embodiment of processor unit 120. Processor unit 120B has a configuration for estimating chemical structures. Therefore, processor unit 120B can also be the processor unit of chemical structure estimation system 1000.

The processor unit 120B includes a first generating means 126, a second generating means 127, a derivation means 122, and an output means 123. In FIG. 4C, the same reference numerals are used to designate components similar to those described above with reference to FIG. 4A, and detailed descriptions thereof will be omitted here.

The first generating means 126 is configured to convert the chemical structure to generate an image. The first generating means 126 has a configuration similar to that of the generating means 124 described above with reference to FIG. 4B. That is, the first generating means 126 can be realized using an MTM converter.

The generated image is passed to the second generation means 127.

The second generating means 127 is configured to modify the generated image to generate a modified image.

The second generating means 127 can perform any modification to the image. For example, the second generating means 127 can add, subtract, multiply, or divide a given value to the real value or pixel value of any pixel in the image.

Alternatively, if the first generating means 126 converts multiple chemical structures to generate multiple images, the second generating means 126 can perform the modification by combining the multiple images. The combination of the multiple images can be, for example, averaging the multiple images. This allows each pixel value of the generated image (i.e., the modified image) to be the average or weighted average of the corresponding pixel values of each of the multiple images. Alternatively, each pixel value of the modified image can be the average or weighted average of the real values of the corresponding pixels of each of the multiple images.

The derivation means 122 is configured to derive a character string corresponding to the altered image. The derived character string is passed to the output means 123.

The output means 123 is configured to output a chemical structure corresponding to the derived character string.

Since the processor unit 120B outputs a character string by modifying an image converted from an input chemical structure, it is possible to output a character string with a chemical structure different from the input chemical structure. For example, when two chemical structures are input, a character string with a chemical structure that may be the average of the two chemical structures is output. For example, when one chemical structure is input, a character string with a modified chemical structure is output. This allows the processor unit 120B to estimate a new chemical structure.

Figure 4D shows an example of the configuration of processor unit 120C, which is an alternative embodiment of processor unit 120. Processor unit 120C has a configuration for estimating chemical structures with modified (preferably improved) properties. Thus, processor unit 120C can also be the processor unit of chemical structure estimation system 1000.

The processor unit 120C includes a first generating means 126, a second generating means 127, a derivation means 122, a third generating means 128, a prediction means 125, and a determination means 129. In FIG. 4D, the same reference numbers are used for configurations similar to those described above with reference to FIG. 4A, and detailed descriptions thereof are omitted here.

The first generating means 126 is configured to convert the chemical structure to generate an image.

The second generating means 127 is configured to modify the generated image to generate a modified image.

The deriving means 122 is configured to derive a plurality of sets of character strings corresponding to the altered image. The deriving means 122 can derive the plurality of sets of character strings using a T-sampling method. Each of the plurality of sets of character strings may correspond to the altered image, but the degree of correspondence varies. The derived plurality of sets of character strings are passed to a third generating means 128.

The third generating means 128 is configured to convert each of the multiple sets of character strings to generate multiple images. The third generating means 128 has a configuration similar to that of the generating means 124 described above with reference to FIG. 4B. The third generating means 128 processes each of the multiple sets of character strings to generate each image. For example, the third generating means 128 generates a first image from a first character string, a second image from a second character string, a third image from a third character string, ... and an nth image from an nth character string. As described above, each of the multiple sets of character strings may correspond to an altered image, but since the correspondences vary, the generated images may not match the altered image, but the generated images may be similar to the altered image.

The prediction means 125 is configured to predict the characteristics of the chemical structure corresponding to each of the multiple images generated, based on each of the multiple images. The prediction means 125 can predict the characteristics of each of the multiple images by processing each of the multiple images. For example, the prediction means 125 predicts the characteristics of a corresponding first chemical structure from a first image, predicts the characteristics of a corresponding second chemical structure from a second image, predicts the characteristics of a corresponding third chemical structure from a third image, ... predicts the characteristics of a corresponding nth chemical structure from an nth image.

The identification means 129 is configured to identify a chemical structure with an altered property based on the predicted property. The identification means 129 can compare the properties of the chemical structures corresponding to each of the multiple images and identify a chemical structure that satisfies a predetermined criterion. The predetermined criterion can be, for example, a chemical structure having the highest or lowest property value, a chemical structure having a property value higher or lower than the input chemical structure, etc.

The processor unit 120C can identify a chemical structure having modified properties among chemical structures obtained by modifying an input chemical structure. For example, when two chemical structures are input, a chemical structure having modified properties can be identified among chemical structures that can be the average of the two chemical structures. For example, when one chemical structure is input, a chemical structure having modified properties can be identified among modified chemical structures. This allows the processor unit 120B to estimate a new chemical structure with modified properties.

In the above example, the target was a chemical structure, but the target is not limited to a chemical structure. The target can be any object as long as it has a structure and the structure is uniquely associated with an image.

Each component of the system 100 described above may be composed of a single hardware part, or may be composed of multiple hardware parts. When composed of multiple hardware parts, the manner in which the hardware parts are connected does not matter. The hardware parts may be connected wirelessly or by wire. The system 100 of the present invention is not limited to a specific hardware configuration. It is also within the scope of the present invention that the processor unit 120 is configured by an analog circuit rather than a digital circuit. The configuration of the system 100 of the present invention is not limited to the one described above as long as it can realize its functions.

3. Processing in image-to-character string conversion system 100 Fig. 6 shows an example of processing (process 600) in image-to-character string conversion system 100. Process 600 is processing for converting an image into a character string. Process 600 can be executed in, for example, processor unit 120. In the following, the processor unit 120 will be described as executing process 600, but processor unit 120A can also execute process 600 in a similar manner.

In step S601, the receiving means 121 of the processor unit 120 receives an image. The image may be any image that can be associated with a character string. Preferably, the image is an image representing a chemical structure (e.g., MTM). The received image is passed to the derivation means 122.

In step S602, the derivation means 122 of the processor unit 120 derives a character string corresponding to the image.

The derivation means 122 can derive a character string corresponding to an image, for example, by using a trained model. Here, the trained model has been trained to output the character following that character when a training image and a character that is part of a character string corresponding to the training image are input. As a result, when an image and a character that is part of a character string corresponding to the image are input to the trained model, the character following that character in the string is output.

For example, when the image received in step S601 and the first character in the string corresponding to that image (e.g., the character corresponding to the Start token in SMILES) are input to the trained model, the second character in the string is output from the trained model. For example, when the image received in step S601, the first character in the string corresponding to that image, and the second character previously output are input to the trained model, the third character in the string is output from the trained model. By repeating this, all characters in the string are output.

When the trained model outputs the next character in the string, the trained model can, for example, output the probability of each of a plurality of candidate characters that are candidates for the next character being the next character, and output the next character based on the probability. In one embodiment, one of the plurality of candidate characters with the highest probability can be selected, and the selected candidate character can be output as the next character. This corresponds to the Argmax method described above. In another embodiment, one of the plurality of candidate characters can be selected according to the distribution of the probabilities of each of the plurality of candidate characters, and the selected candidate character can be output as the next character. This corresponds to the T-sampling method described above. The distribution of probabilities can be weighted, and can be adjusted to obtain a desired result. The weight corresponds to the value of T in the T-sampling method described above. The larger T is, the smaller the difference in probability between each candidate character is, and the candidate characters are selected with a more equal probability.

The string derived by repeatedly outputting the characters in the string by selecting one of multiple candidate characters according to a probability distribution will not necessarily be the same when process 600 is performed multiple times on the same image. This is because there will be variation in the derived string according to the probability distribution. This makes it possible to derive multiple candidate strings that may correspond to a single image from that image.

For example, in the above-mentioned example of SMILES, the trained model outputs the probability that each of multiple candidate characters (32 candidate characters corresponding to the SMILES token) that are candidates for the next character will be the next character. The derivation means 122 may select the candidate character with the highest probability (one candidate character corresponding to the SMILES token) from the outputs from the trained model, or may select one candidate character (one candidate character corresponding to the SMILES token) according to the probability distribution.

In step S603, the output means 123 of the processor unit 120 outputs the character string derived in step S602. The output means 123 can, for example, convert the derived character string into information that is easily understandable to the user and output it. For example, if the derived character string represents a chemical structure, the output means 123 can convert the derived character string into a structural formula and output it.

The output string can be used for any purpose. In one example, the output string can be used to predict properties of an object.

For example, after step S603, the generation means 124 of the processor unit 120A can convert the output character string to generate an image, and the prediction means 125 of the processor unit 120A can predict the characteristics of the object represented by the image based on the generated image.

The prediction means 125 can predict the characteristics using a second trained model that has learned the relationship between the training image and the characteristics of the object represented by the training image. When an image is input, the second trained model can output the characteristics of the object represented by the image.

In one example, the outputted string representing the chemical structure is converted to an MTM, and the converted MTM is input to a second trained model, which can predict properties of the chemical structure.

Figure 7 shows an example (700) of a process in the chemical structure estimation system 1000. Process 700 is a process for estimating a chemical structure. Process 700 can be executed, for example, in the processor unit 120B.

In step S701, the first generating means 126 of the processor unit 120B converts the chemical structure to generate an image. The first generating means 126 can convert the chemical structure to generate an image (MTM) using, for example, an MTM converter.

In step S701, the first generating means 126 can generate a plurality of images by converting each of the plurality of chemical structures. For example, the first generating means 126 can convert a first character string representing a first chemical structure to generate a first image, and convert a second character string representing a second chemical structure to generate a second image.

In step S702, the second generating means 127 of the processor unit 120B generates an altered image by modifying the image generated in step S701.

The second generating means 127 can perform any modification to the generated image by converting the chemical structure. For example, the second generating means 127 can add, subtract, multiply, or divide a given value to the real value or pixel value of any pixel in the image.

In the case where multiple images are generated by converting multiple chemical structures, the second generating means 127 can generate the modified image by combining the multiple images. The combination of the multiple images can be, for example, averaging the multiple images. In this way, each pixel value of the generated image (i.e., the modified image) can be the average or weighted average of the corresponding pixel values of each of the multiple images. Alternatively, each pixel value of the modified image can be the average or weighted average of the real values of the corresponding pixels of each of the multiple images.

In step S703, the derivation means 122 of the processor unit 120B derives a character string corresponding to the altered image. The derivation means 122 can derive the character string in a manner similar to step S602.

For example, when the altered image generated in step S702 and the first character in the string corresponding to the altered image (e.g., the character corresponding to the Start token in SMILES) are input to the trained model, the second character in the string is output from the trained model. For example, when the altered image generated in step S702, the first character in the string corresponding to the altered image, and the second character previously output are input to the trained model, the third character in the string is output from the trained model. By repeating this, all characters in the string are output.

The derivation means 122 may use a probability distribution to derive multiple character string candidates.

In step S704, the output means 123 of the processor unit 120B outputs a chemical structure corresponding to the derived character string. The output means 123 can output the chemical structure as a structural formula, for example.

In this way, a new chemical structure can be estimated from the input chemical structure. If multiple chemical structures are input, a chemical structure that may be the average of the multiple chemical structures can be estimated. In the field of drug discovery, this can lead to the creation of new drugs.

Figure 8 shows an example of a process (800) in the chemical structure estimation system 1000. Process 800 is a process for estimating a chemical structure with modified properties. Process 800 can be executed, for example, in the processor unit 120C.

In step S801, the first generating means 126 of the processor unit 120C converts the chemical structure to generate an image. The first generating means 126 can convert the chemical structure to generate an image (MTM) using, for example, an MTM converter.

In step S801, the first generating means 126 can generate a plurality of images by converting each of the plurality of chemical structures. For example, the first generating means 126 can convert a first character string representing a first chemical structure to generate a first image, and convert a second character string representing a second chemical structure to generate a second image.

In step S802, the second generating means 127 of the processor unit 120C generates an altered image by modifying the image generated in step S802.

The second generating means 127 can perform any modification to the generated image by converting the chemical structure. For example, the second generating means 127 can add, subtract, multiply, or divide a given value to the real value or pixel value of any pixel in the image.

In the case where multiple images are generated by converting multiple chemical structures, the second generating means 127 can generate the modified image by combining the multiple images. The combination of the multiple images can be, for example, averaging the multiple images. In this way, each pixel value of the generated image (i.e., the modified image) can be the average or weighted average of the corresponding pixel values of each of the multiple images. Alternatively, each pixel value of the modified image can be the average or weighted average of the real values of the corresponding pixels of each of the multiple images.

In step S803, the derivation means 122 of the processor unit 120C derives a plurality of sets of character strings corresponding to the altered image generated in step S802. The derivation means 122 can derive character strings using the trained model in a manner similar to step S602.

For example, when the altered image generated in step S802 and the first character in the string corresponding to the altered image (e.g., the character corresponding to the Start token in SMILES) are input to the trained model, the second character in the string is output from the trained model. For example, when the altered image generated in step S702, the first character in the string corresponding to the altered image, and the second character previously output are input to the trained model, the third character in the string is output from the trained model. By repeating this, all characters in the string are output.

The derivation means 122 derives multiple sets of character strings by repeating the process of deriving character strings corresponding to the altered image. At this time, by deriving character strings using the T-sampling method, the multiple sets of character strings derived by the repeated process will vary.

In step S804, the third generating means 128 of the processor unit 120C converts each of the sets of character strings derived in step S803 to generate a plurality of images. The third generating means 128 can convert the chemical structure to generate an image (MTM) using, for example, an MTM converter. The third generating means 128 generates each image by performing processing on each of the sets of character strings. For example, the third generating means 128 generates a first image from the first character string, a second image from the second character string, a third image from the third character string, ... and an nth image from the nth character string.

In step S805, the prediction means 125 of the processor unit 120C predicts the characteristics of each of the multiple images generated in step S804, based on each of the multiple images. The prediction means 125 can predict the characteristics using a second trained model that has learned the relationship between the learning image and the characteristics of the object represented by the learning image. When an image is input, the second trained model can output the characteristics of the object represented by the image. The prediction means 125 can predict the characteristics of each of the multiple images by performing processing on each of the images. For example, the prediction means 125 predicts the characteristics of a corresponding first chemical structure from a first image, predicts the characteristics of a corresponding second chemical structure from a second image, predicts the characteristics of a corresponding third chemical structure from a third image, ... predicts the characteristics of a corresponding nth chemical structure from an nth image.

In step S806, the identification means 129 of the processor unit 120C identifies a chemical structure whose property has been modified based on the property predicted in step S805. The identification means 129 can compare multiple properties and identify a chemical structure whose property satisfies a predetermined criterion. The predetermined criterion can be, for example, a chemical structure having the highest or lowest property value, a chemical structure having a property value higher or lower than the input chemical structure, etc.

In this way, a chemical structure with modified (e.g., improved) properties can be inferred from an input chemical structure. When multiple chemical structures are input, a chemical structure with modified properties common to the multiple chemical structures can be inferred. This can lead to the creation of novel and effective drugs in the field of drug discovery.

In the above example, the target was a chemical structure, but the target is not limited to a chemical structure. The target can be any object as long as it has a structure and the structure is uniquely associated with an image.

In the examples described above with reference to Figures 6 to 8, the processes are described to be performed in a specific order, but the order of each process is not limited to that described, and the processes may be performed in any order that is logically possible.

In the examples described above with reference to Figures 6 to 8, it has been explained that the processing of each step shown in Figures 6 to 8 is realized by processor unit 120, processor unit 120A, processor unit 120B, or processor unit 120C and a program stored in memory unit 130, but the present invention is not limited to this. At least one of the processing of each step shown in Figures 6 to 8 may be realized by a hardware configuration such as a control circuit.

The present invention is not limited to the above-described embodiment. It is understood that the scope of the present invention should be interpreted only by the claims. It is understood that a person skilled in the art can implement an equivalent scope based on the description of the present invention and common technical knowledge from the description of the specific preferred embodiment of the present invention. For example, it is naturally understood that the features described for one embodiment can also be applied to another embodiment.

(Construction of MTM inverter)
An MTM inverter was constructed for Bromodomain-containing protein 4 (BRD4) inhibitors. As a data set for construction, the chemical structures and Ki values of BRD4 inhibitors were obtained from ChEMBL (https://www.ebi.ac.uk/chembl/). For duplicated chemical structures, the average Ki value was used as the representative value, and duplicates were removed by leaving one chemical structure. As a result, 839 chemical structures and their Ki values were obtained. Here, these 839 pieces of data will be referred to as the BRD4 data set.

In the BRD4 dataset, as shown below, information on one chemical structure is described in each row, specifically, SMILES compound ID/pKi value is listed. The pKi value is a value obtained by pKi = -log(Ki), and the larger the value, the stronger the inhibitory activity.
Example: BRD4 dataset:
Cc1[nH]c(-c2cc(N)ccc2Oc2ccccc2)cc1C(N)=O CHEMBL4088536/6.38
COc1cc2c(cc1-c1c(C)noc1C)[nH]c1ncnc(Cl)c12 CHEMBL4209208/6.365
Cc1cc(C)c(S(=O)(=O)O)cc1/N=N/c1cc(C)c(O)c(C)c1 CHEMBL3087050/5.255

The BRD4 dataset was split into training data (671 entries) and test data (168 entries) in a ratio of 8:2.

The MTM inverter disclosed herein was constructed using chemical structures contained in the training data (671 pieces).

(Accuracy evaluation of MTM inverter)
The conversion accuracy of the MTM inverter was evaluated using training data (671 items) and test data (168 items).

The conversion accuracy was evaluated using the following two conversion methods and two evaluation indices.
Conversion methods: Argmax method, T-Sampling method Evaluation indices: Reconstruction, SMILES validity, Similarity

Reconstruction is an index showing whether the chemical structure converted back from MTM matches the original chemical structure. Chemical structures are expressed in SMILES. SMILES match was determined by converting SMILES to Canonical SMILES using RDKit (https://www.rdkit.org/) and checking for string match. Canonical SMILES refers to a standardized SMILES that is converted so that one chemical structure is converted to one SMILES.

The validity of SMILES is an index showing whether SMILES converted from MTM is grammatically correct. The grammatical correctness of SMILES was determined using the Chem. MolFromSmiles function of the RDKit (https://www.rdkit.org/).

Similarity is an index showing the similarity between a chemical structure reverse-converted from an MTM and the original chemical structure. The MTM of a chemical structure reverse-converted from an MTM is converted using an MTM converter. The similarity between chemical structures is defined as the similarity between MTMs, and is calculated using the Tanimoto coefficient (real number version) (Int J Data Sci Anal 4, 153-172 (2017). https://doi.org/10.1007/s41060-017-0064-z).

The evaluation results for each of the Argmax method and the T-Sampling method are shown in Table 1.

Reconstruction of the training data using the Argmax method was 78.2%, and reconstruction using the T-sampling method was 97.8%. The accuracy of reconstruction is significantly improved by the T-sampling method. Validity was 96.3% for the Argmax method and 100% for the T-sampling method. Average similarity was 0.990 for the Argmax method and 0.998 for the T-sampling method.

Next, the reconstruction rate using the Argmax method for the test data was 10.1%, and the reconstruction rate using the T-sampling method was 30.4%. Even for the test data, the accuracy of reconstruction was significantly improved by the T-sampling method. The validity was 84.5% for the Argmax method and 99.4% for the T-sampling method. The average similarity was 0.906 for the Argmax method and 0.956 for the T-sampling method. The reconstruction rate was 30.4%, but as can be seen from the average similarity, it is clear that a chemical structure very similar to the original chemical structure was generated.

The two parameters required for the T-sampling method are temperature and the number of samplings. The set values are shown in Table 2.

Here, 100 samples were taken for each T-sampling. As a result, the average validity was 85.4% for the training set and 70.4% for the test set. The average uniqueness was 6.5% for the training set and 9.9% for the test set. Uniqueness refers to chemical structures that do not have duplicates. If 10 unique chemical structures are generated through 100 samples, the uniqueness will be 10.0%.

Figure 9 shows the distribution of similarities in the training data and test data. It can be seen that the T-sampling method significantly increases the number of chemical structures similar to the original chemical structure. From these results, it was confirmed that inverse conversion using the MTM inverter can be performed with an accuracy of 30.4% in reconstruction and 0.956 in average similarity.

(Estimation of novel BRD4 inhibitors using MTM inverters)
From known BRD4 inhibitors, BRD4 inhibitors with improved biological activity were predicted using MTM inverters, the flow of which corresponds to the flow shown in Figure 1B.

From the training data, 1000 pairs of BRD4 inhibitors with high activity (pKi value > 8.0) were randomly selected. The compounds selected here are called base compounds, and the pairs are called base compound pairs.

The base compound pair consists of base compound A (C11) and base compound B (C22).

The mixed MTM (I13) is calculated from the MTM (I11) of the base compound A and the MTM (I12) of the base compound B according to the following formula.
0<λ<1, typically λ=0.5, which means that the mixed MTM (I13) is just the average of the MTM (I11) of base compound A and the MTM (I12) of base compound B.

Using an MTM inverter, MTM (I13) was reverse-converted to a chemical structure (SMILES) using the T-sampling method. Here, the temperature (Temperature) in the T-sampling method was set to 1.5, and the number of samplings was set to 100. Multiple sets of SMILES were obtained by reverse conversion using the T-sampling method. The obtained SMILES were converted to MTM using an MTM converter. The chemical structure (SMILES) that is the source of MTM and is most similar to MTM (I13) was presented as the chemical structure obtained by reverse conversion of MTM (I13).

To predict activity for BRD4, a BRD4 activity prediction model was used. The BRD4 activity prediction model corresponds to the second trained model described above. MTM (I13) was used as input, and activity for BRD4 was predicted using the BRD4 activity prediction model. In addition, MTM converted from SMILES obtained using an MTM inverter using an MTM converter was used as input, and activity for BRD4 was predicted using the BRD4 activity prediction model.

The BRD4 activity prediction model used the same training data as when constructing the MTM converter.

The following four hyperparameters were optimized using the automatic hyperparameter optimization framework optuna (https://github.com/optuna/optuna) for the DNN parameters:
・Conv2D layer argument filters of conv2d_1
・Conv2D layer argument filters of conv2d_2
・Dense layer argument units for dense_1
・Dropout layer argument rate of dropout_1

Finally, the hyperparameters of the DNN were as follows:
・The arguments for the Conv2D layer of conv2d_1 are filters 48 and activation is “relu”
・The arguments for the Conv2D layer of conv2d_2 are filters 96 and activation is “relu”
・The Dense layer argument units for dense_1 is 512, and activation is "relu".
・The rate argument of the Dropout layer for dropout_1 is 0.2
・The activation argument for the Dense layer in dense_2 is “liner”
・The loss function in DNN is “mean_squared_error”
・The optimization algorithm for the loss function is "adam"
・Number of epochs is 100
・Batch size is 16

The BRD4 activity prediction model was evaluated using the following three evaluation indexes.
Mean Squared Error (MSE)
・ Mean Absolute Error (MAE)
Coefficient of determination (R2):

The results are shown in Table 3.

We confirmed good prediction accuracy on the test data, with MSE = 0.277, MAE = 0.389, and R2 = 0.755.

Compounds satisfying all three conditions were selected from 1000 chemical structures generated from 1000 sets of base compounds using the constructed MTM inverter and BRD4 activity prediction model.
Condition 1: The predicted pKi of MTM (I13) is greater than the measured pKi of both base compounds A and B.
Condition 2: The predicted value (pKi) of MTM obtained from SMILES converted by the MTM inverter is greater than the measured values (pKi) of both base compounds A and B.
Condition 3: The chemical structure of SMILES converted by the MTM inverter is not included in the BRD4 dataset.

As a result, 25 chemical structures were selected as BRD4 inhibitors. The compounds with the best predictive values are shown in Figure 10.

The 25 selected compounds could be classified into the following four types (Figures 11A and 11B).

Here, base compound A is called compound A, base compound B is called compound B, and the compound represented by SMILES converted by the MTM inverter is called compound C.
Type A:
Compound C satisfies all of the following conditions:
- Compound C is compound A and MMP.
- Compound C is compound B and MMP.
However, Compound A and Compound B are not MMPs.
Type B:
Compound C satisfies all of the following conditions:
- Compound C is compound A and MMP.
- Compound C is compound B and MMP.
However, compound A and compound B are MMPs.
Type C:
Compound C satisfies one of the following conditions:
- Compound C is compound A and MMP.
- Compound C is compound B and MMP.
Type D:
Compound C satisfies all of the following conditions:
Compound C is not compound A or an MMP.
Compound C is not an MMP like compound B.

Here, MMP (Matched Molecular Pair) refers to a pair of molecules that share a common structure for the most part, but differ in only one partial structure.

In the field of drug discovery, synthesis is expanded with a focus on compounds that will become MMPs for lead compounds, and optimization is carried out to produce compounds suitable for pharmaceuticals. Among the novel compounds estimated using the MTM inverter and BRD4 activity prediction model disclosed herein, there were several compounds that have an MMP relationship with base compound A and/or base compound B. Therefore, it is suggested that the method disclosed herein may also be useful in the field of drug discovery.

Generation of Novel and Active BRD4 Inhibitors
From the 1,000 pairs, compounds that satisfied all of the following conditions were selected as BRD4 inhibitors.

Condition: The reverse-converted chemical structure is included in the test data.

The MTM inverter is trained based on the chemical structures of the training data, so it does not contain any chemical structures of the test data. If the test data contains chemical structures reverse-converted from the MTM (I13) of 1,000 sets of base compounds, this indicates that a novel and active BRD4 inhibitor has been generated.

The results were as follows:
New chemical structures: 485 (chemical structures not included in the BRD dataset)
Existing chemical structures: 515 (chemical structures included in the BRD dataset)
In training data: 485
In validation data: 10
In the test data: 20
Total 1000

Of the 1,000 compounds generated, 20 were included in the test data. However, because they contained duplicate chemical structures, after removing them, eight compounds remained. As a result, eight compounds that were actually active against BRD4 were generated.

The present invention is useful for providing a system for converting an image into a string of characters, specifically, a system for converting an image showing a chemical structure into a string of characters showing the chemical structure.

100 Image-character string conversion system 110 Interface section 120, 120A, 120B, 120C Processor section 130 Memory section 200 Database section 300 Terminal device 400 Network 1000 Chemical structure estimation system

Claims (25)

A receiving means for receiving an image;
A derivation means for deriving a character string corresponding to the image;
and an output means for outputting the derived string. The system according to claim 1, wherein the derivation means, when a learning image and a character in a string corresponding to the learning image are input, uses a trained model that has been trained to output the character next to the input character, to derive a string corresponding to the image. The trained model is
outputting a probability of each of a plurality of candidate characters being the next character;
and outputting the next character based on the probability. outputting the next character based on the probability
selecting one candidate character from the plurality of candidate characters that has the highest probability; and
and outputting the selected candidate character as the next character. outputting the next character based on the probability
selecting a candidate character from the plurality of candidate characters according to the probability distribution for each of the plurality of candidate characters;
and outputting the selected candidate character as the next character. The system of claim 5, further comprising weighting the distribution of probabilities. The system of claim 1, wherein the image is an image representing a chemical structure. The system of claim 7, wherein the character string is a character string representing a chemical structure. The system according to claim 8, wherein the output means converts the derived character string into a structural formula and outputs it. A generating means for converting a character string to generate an image;
A prediction means for predicting a characteristic of the chemical structure by using an image obtained by converting a character string representing the training chemical structure and a second trained model that has learned a relationship between the image and the characteristic of the training chemical structure,
The generating means converts the output character string to generate an image;
The system according to any one of claims 7 to 9, wherein the prediction means predicts, based on the generated image, a property of a chemical structure corresponding to the image. A chemical structure prediction system, comprising:
A first generating means for converting a chemical structure to generate an image;
A second generating means for generating an altered image by altering the generated image;
A derivation means for deriving a character string corresponding to the altered image;
and an output means for outputting a chemical structure corresponding to the derived character string. the first generating means converts a first character string representing a first chemical structure to generate a first image, and converts a second character string representing a second chemical structure to generate a second image;
The system of claim 11 , wherein the second generating means generates the altered image by combining the first image and the second image. A system for predicting a chemical structure having an altered property, comprising:
A first generating means for converting a chemical structure to generate an image;
A second generating means for generating an altered image by altering the generated image;
A derivation means for deriving a plurality of sets of character strings corresponding to the altered image;
a third generating means for generating a plurality of images by transforming each of the derived sets of character strings;
A prediction means for predicting, based on each of the plurality of images, a property of a chemical structure corresponding to each image;
and an identification means for identifying a chemical structure having an altered property based on the predicted property. The system of claim 13, wherein the property is an activity associated with the chemical structure. The system of claim 14, wherein the activity is a biological or chemical activity possessed by a compound corresponding to the chemical structure. The system of claim 15, wherein the activity is an enzyme inhibitory activity. A system for predicting a structure of an object having an altered characteristic, comprising:
a first generating means for generating an image uniquely associated with the structure of interest;
a second generating means for generating an altered image by altering the generated image;
A derivation means for deriving a plurality of sets of character strings corresponding to the altered image;
a generating means for generating a plurality of images by converting each of the derived sets of character strings;
A prediction means for predicting characteristics corresponding to each of the plurality of images based on each of the images;
and an identification means for identifying a structure of the object having an altered property based on the predicted property. Receiving an image; and
deriving a character string corresponding to the image; and
and outputting the derived string. A program, the program being executed in a computer having a processor unit, the program comprising:
Receiving an image; and
deriving a character string corresponding to the image; and
and outputting the derived character string. A chemical structure prediction method, comprising the steps of:
Transforming the chemical structure to generate an image;
modifying the generated image to generate an altered image;
deriving a character string corresponding to the altered image;
and outputting a chemical structure corresponding to the derived string. A chemical structure prediction program, the program being executed on a computer having a processor unit, the program comprising:
Transforming the chemical structure to generate an image;
modifying the generated image to generate an altered image;
deriving a character string corresponding to the altered image;
and outputting a chemical structure corresponding to the derived character string. A method for predicting a chemical structure with modified properties, comprising:
Transforming the chemical structure to generate an image;
modifying the generated image to generate an altered image;
deriving a plurality of sets of character strings corresponding to the altered image;
transforming each of the derived sets of strings to generate a plurality of images;
predicting, based on each image of the plurality of images, a property of a chemical structure corresponding to the respective image;
and identifying a chemical structure having an altered property based on the predicted property. A program for predicting a chemical structure having modified properties, the program being executed on a computer having a processor unit, the program comprising:
Transforming the chemical structure to generate an image;
modifying the generated image to generate an altered image;
deriving a plurality of sets of character strings corresponding to the altered image;
transforming each of the derived sets of strings to generate a plurality of images;
predicting, based on each image of the plurality of images, a property of a chemical structure corresponding to the respective image;
and identifying a chemical structure having an altered property based on the predicted property. A method for predicting the structure of an object having an altered property, comprising:
generating an image uniquely associated with the structure of interest;
modifying the generated image to generate an altered image;
deriving a plurality of sets of character strings corresponding to the altered image;
transforming each of the derived sets of strings to generate a plurality of images;
predicting, based on each image of the plurality of images, a characteristic corresponding to the respective image;
and identifying a structure of the subject having an altered property based on the predicted property. A program for predicting a structure of an object whose characteristics have been modified, the program being executed on a computer having a processor unit, the program comprising:
generating an image uniquely associated with the structure of interest;
modifying the generated image to generate an altered image;
deriving a plurality of sets of character strings corresponding to the altered image;
transforming each of the derived sets of strings to generate a plurality of images;
predicting, based on each image of the plurality of images, a characteristic corresponding to the respective image;
and identifying a structure of the target having an altered property based on the predicted property.