WO2022196301A1 - 候補アイディアを生成するシステム及び方法 - Google Patents

候補アイディアを生成するシステム及び方法 Download PDF

Info

Publication number
WO2022196301A1
WO2022196301A1 PCT/JP2022/008022 JP2022008022W WO2022196301A1 WO 2022196301 A1 WO2022196301 A1 WO 2022196301A1 JP 2022008022 W JP2022008022 W JP 2022008022W WO 2022196301 A1 WO2022196301 A1 WO 2022196301A1
Authority
WO
WIPO (PCT)
Prior art keywords
vector
idea
contracted
vectors
candidate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/008022
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
好弘 刑部
彰規 淺原
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hitachi Ltd
Original Assignee
Hitachi Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hitachi Ltd filed Critical Hitachi Ltd
Priority to EP22771057.1A priority Critical patent/EP4310740A4/en
Priority to US18/272,233 priority patent/US12475195B2/en
Publication of WO2022196301A1 publication Critical patent/WO2022196301A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0475—Generative networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G06N5/022—Knowledge engineering; Knowledge acquisition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks

Definitions

  • the present invention relates to a system for generating candidate ideas.
  • a system that aims to improve system efficiency and intellectual productivity by digitizing and quantitatively analyzing real-world observation data in cyberspace is called a cyber-physical system (CPS).
  • CPS cyber-physical system
  • system optimization using CPS is realized by alternately performing a phase of planning improvement measures for the current situation and a phase of observing what actually happens when the improvement measures are taken.
  • Bayesian optimization is a method of determining the next idea to try based on expected values calculated from known information.
  • Japanese Patent Application Laid-Open No. 2020-95452 discloses a technique using Bayesian optimization to search for candidate compounds to be tested next in material development.
  • One aspect of the present invention is a system for generating candidate ideas to be evaluated next, comprising: one or more processors; one or more storage devices storing programs executed by the one or more processors; One or more processors convert a first input idea vector representing an idea into a first reduced vector, in the multivariate space to which the first reduced vector belongs, and in the multivariate space of the first reduced vector generating one or more second contracted vectors based on the values of the first predetermined components, and from each of the one or more second contracted vectors, one or more first output idea vectors representing candidate ideas to be presented to a user;
  • the multivariate space generates a similarity between an input idea vector generating a contracted vector and an output idea vector generated from the contracted vector, a first predetermined component of the contracted vector and the first
  • the similarity with the predetermined index value is configured to be maintained.
  • candidate ideas to be evaluated in the next trial can be generated based on the novelty and quality of the candidate ideas.
  • FIG. 1 illustrates an example logical configuration of a candidate idea suggestion system according to one embodiment herein.
  • 1 shows an example hardware configuration of a candidate idea generation device, a data storage device, and a user interface;
  • An example of parameter data included in the parameter database is shown.
  • 4 shows a configuration example of an idea expression database.
  • a configuration example of an idea vector database is shown.
  • FIG. 4 is a flowchart of example processing of an idea vectorization module;
  • FIG. An example of generating a structural formula vector, which is an idea vector, from a chemical structural formula, which is an idea expression, will be shown.
  • a configuration example of a contracted vector database is shown.
  • FIG. 11 is a flow chart showing a processing example of an idea reduction module;
  • FIG. 1 shows a configuration example of a variational autoencoder.
  • FIG. 11 is a flow chart showing a processing example of a contracted space search module;
  • FIG. 4 shows a configuration example of a candidate idea database.
  • FIG. 4 is a flow chart of example processing of a candidate idea generation module;
  • FIG. 11 shows an example of a user interface (screen) of an idea support application according to the second embodiment;
  • FIG. 11 shows an example of a user interface (screen) of an idea support application according to the second embodiment;
  • Here is an example of a word vector.
  • This system may be a physical computer system (one or more physical computers), or a system built on a computing resource group (multiple computing resources) such as a cloud platform.
  • a computer system or a group of computing resources includes one or more interface devices (for example, including communication devices and input/output devices), one or more storage devices (for example, including memory (main storage) and auxiliary storage devices), and one or more processor.
  • a function described with a function as the subject may be a process performed by a processor or a system having the processor.
  • Programs may be installed from program sources.
  • the program source may be, for example, a program distribution computer or a computer-readable storage medium (eg, a computer-readable non-transitory storage medium).
  • the description of each function is an example, and multiple functions may be combined into one function, or one function may be divided into multiple functions.
  • a report (document) in R&D strategy planning is a form of idea that indicates the guidelines for the next R&D.
  • Sentences contained in a document, or vector representations obtained after morphological analysis is performed on sentences, are also forms of ideas.
  • FIG. 1 shows an example of the logical configuration of a candidate idea proposal system according to one embodiment of the present specification.
  • the candidate idea proposal system includes a candidate idea generation device M01, a data storage device M02, and a user interface M05. They can communicate with each other via a network.
  • the candidate idea generation device M01 generates candidate ideas to be proposed to the user.
  • the candidate idea generation device M01 includes an idea vectorization module P01, an idea reduction module P02, a reduced space search module P03, and a candidate idea generation module P04.
  • the data storage device M02 stores data referenced or generated by the candidate idea generation device M01.
  • the data storage device M02 stores various databases. Specifically, an idea expression database DB01, an idea vector database DB02, a contracted vector database DB03, a candidate idea database DB04, a model database DB05, and a parameter database DB06 are stored.
  • the data storage device M02 further manages databases DB01 to DB06. Includes data management module P06.
  • the user interface M05 allows the user to enter data and present information to the user.
  • the user interface M05 includes a display portion P07 and an input reception portion P08.
  • the display unit P07 presents information generated by the candidate idea generation device M01 to the user, for example.
  • the input reception unit P08 receives, for example, information necessary for the candidate idea generation device M01 to perform processing from the user.
  • the idea vectorization module P01 of the candidate idea generation device M01 converts an idea expression expressed in a predetermined format into a real number vector, ie, an idea vector, according to a predetermined method.
  • the idea contraction module P02 converts the idea vector from the idea vectorization module P01 into another real number vector, that is, a contracted vector.
  • the idea reduction module P02 converts the similarity of one or more index values preset by the user so as to correspond to the distance in the multivariate space configured by the contracted vector.
  • the idea contraction module P02 transforms a multivariate space formed by contracted vectors so that the similarity of the input idea vector and the similarity of the index value of the quality of the idea are correlated.
  • the idea reduction module P02 can include, for example, one variational autoencoder (VAE).
  • VAE is a type of autoencoder, and is a deep generative model composed of two neural networks, an encoder and a decoder.
  • An encoder transforms an input (vector) into a real vector.
  • the space to which the real vectors belong is called the latent space and is assumed to follow a given distribution, eg a normal distribution.
  • the decoder inverse transforms the real vector and outputs a vector of the same dimension as the input.
  • General encoders and decoders are trained (learn) so that their inputs and outputs are equal.
  • the ability to reconstruct the input from the intermediate output real vector means that sufficient features of the input are reflected in the real vector.
  • the dimension of the latent space is set to be smaller than the dimension of the input. Therefore, the encoder can both extract the features of the input and compress the dimensionality of the input.
  • the intermediate output vector is called a latent variable or latent vector, and is an abstract expression representing features extracted from the input idea vector. Therefore, the latent vector of VAE is a contracted vector.
  • the latent vector is assumed to follow a predetermined distribution, eg a Gaussian distribution. Therefore, the decoder can reconstruct the input vector with high accuracy, for example, if it receives a vector with added noise.
  • VAE has high robustness as a generative model.
  • the parameters of the neural network are sequentially optimized so that the output value (loss) of the loss function becomes smaller.
  • it is trained to reduce the error between input and output, ie reconstruction error.
  • a loss function that adds another term to the reconstruction error.
  • the loss function includes a term that indicates the degree of similarity between one or more predetermined components in the latent vector and a predetermined index value, for example, an index value representing the quality of the idea.
  • a term representing the degree of similarity can be, for example, a term representing a correlation, a term representing a Euclidean distance from certain data, or the like.
  • a term is added to the loss function that reduces the loss as the value representing the correlation increases.
  • the VAE latent space trained using this loss function maintains both the similarity between the input idea vector and the corresponding output vector, and the similarity between a given component in the latent vector and a given index value. is a multivariate space
  • the loss function by adding a term according to the search purpose of the candidate idea to the loss function, it is possible to construct a latent space that reflects the desired degree of similarity in the VAE. For example, if the index value (true value) of the actual quality of candidate ideas proposed by this system in the past is known, the difference between the quality (predicted value) of the idea predicted at the time of proposal and the true value of the index value is An error can be added to the loss function. As a result, it is possible to construct a latent space that reflects the degree of similarity between the predicted value and the true value, so that it is possible to avoid regions in which the prediction is likely to be greatly deviated in advance.
  • the contracted space search module P03 searches a target value, for example, the vicinity of a contracted vector corresponding to a known idea with the highest quality at the time of searching, based on the gradient method, in a multivariate space configured by contracted vectors. , we can find a more desirable contraction vector.
  • a target value for example, the vicinity of a contracted vector corresponding to a known idea with the highest quality at the time of searching, based on the gradient method, in a multivariate space configured by contracted vectors.
  • a target value for example, the vicinity of a contracted vector corresponding to a known idea with the highest quality at the time of searching, based on the gradient method, in a multivariate space configured by contracted vectors. , we can find a more desirable contraction vector.
  • given components in the latent vector are correlated with the value of the idea quality indicator, so modifying the given component in a given way allows a gradient-based search along that axis. can be realized. For example, a predetermined value is added to or subtracted from
  • the candidate idea generation module P04 receives the contracted vector discovered by the contracted space search module P03 and converts it into an output format set in advance by the user. For example, the contracted vector is first transformed back to the idea vector and then back to the idea representation. As a result, the contracted vector is restored to an expression format that the user can understand by restoring the same expression format as the input idea expression.
  • An embodiment of the present specification mainly performs the following steps. First, the system receives user settings and data and executes the idea vectorization module P01. The system then learns the VAEs that make up the idea reduction module P02. After the VAE training is completed, the system generates candidate ideas whose quality should be evaluated next by a reduced space search module P03 followed by a candidate idea generation module P04. The generated candidates are presented to the user via the user interface M05.
  • VAE learning must be done first when this system is in operation. For the second and subsequent searches, the learning step may be omitted. Also, when a new trial result of an idea is input, the data may be used for additional learning.
  • the data management module P06 has a function of retrieving stored ideas and a function of accepting and storing actual index values for the quality of candidate ideas.
  • FIG. 2 shows a hardware configuration example of the candidate idea generation device M01, the data storage device M02, and the user interface M05.
  • Candidate idea generator M01 includes a processor U111 having computing power, and a DRAM U112 providing volatile temporary storage for storing programs and data executed by processor U111.
  • the candidate idea generation device M01 further includes a communication device U113 that performs data communication with other devices including other devices in this system, and a permanent information storage area using a HDD (Hard Disk Drive), flash memory, or the like. Auxiliary storage device U114 to provide.
  • the candidate idea generation device M01 also includes an input device U115 that receives operations from the user, and a monitor U116 (an example of an output device) that presents the output results of each process to the user.
  • the candidate idea generation device M01 stores programs such as an idea vectorization module P01, an idea contraction module P02, a contraction space search module P03, and a candidate idea generation module P04.
  • Programs to be executed by the processor U111 and data to be processed are loaded from the auxiliary storage device U114 to the DRAM U112.
  • the hardware elements that make up the data storage device M02 may be the same as those of the candidate idea generation device M01.
  • the data storage device M02 stores various databases DB01 to DB06 and programs such as the data management module P06 shown in FIG. 1, and executes the data management module P06.
  • the hardware elements that make up the user interface M05 may be the same as those of the candidate idea generation device M01.
  • the user interface M05 executes a display part P07 and an input reception part P08.
  • the candidate idea submission system includes one or more storage devices and one or more processors.
  • Example 1 we search for and present the structural formulas of candidate compounds to be tested next.
  • the user can set the parameter database DB06 via the input device of any device.
  • the parameter database DB06 stores data necessary for processing of the candidate idea generation device M01.
  • the parameter database DB06 can contain, for example, information that the idea vectorization module P01 refers to in order to generate an idea vector. As an example, it includes lexical information for converting a chemical structural formula represented by the SMILES notation into a matrix.
  • the parameter database DB0 can also include configuration information for the idea reduction module P02. Specifically, the parameter database DB06 describes the structure and learning parameters of the idea reduction module P02.
  • FIG. 3 shows an example of parameter data contained in the parameter database DB06.
  • TableNames in section 61 indicates a table that stores lexical information for converting chemical structural formulas expressed by SMILES notation into matrices.
  • TigetProperties in section 62 indicates parameters for characteristic values (objective variables) that indicate the quality of the compound. Specifically, characteristic values for mwt and lopP are shown. These characteristic values are referred to in the learning of the idea reduction module P02 and in the search of the reduced vector by the reduced space search module P03.
  • VAEInitialParams in section 63 indicates network configuration parameters of the idea reduction module P02. Since the idea reduction module P02 in this example includes a VAE, configuration parameters for the VAE are shown. For example, “CorrDimIndex” defines which dimension to correlate, and “CorrDir” defines in which direction it is desirable to bias positively or negatively.
  • SearchParams in section 64 indicates parameters that are referred to in searching for reduced vectors by the reduced space search module P03. In this example, the conditions for filtering the proposed candidate reduced vectors are shown.
  • the idea expression database DB01 will be explained.
  • a user can store data in the idea expression database DB01 via the user interface M05.
  • the idea expression database DB01 includes information on past experimental conditions and experimental results.
  • the idea expression database DB01 stores ideas that have been tried so far. What kind of ideas were tried and what kind of KPIs were obtained are associated and accumulated. For example, experimental data corresponds to material development.
  • FIG. 4 shows a configuration example of the idea expression database DB01.
  • the ID column T1C1 represents the identifier of the record.
  • the SuggestID column T1C21 indicates identifiers of candidate ideas proposed by this system in the past. This column T1C21 is associated with the SuggestID column T4C1 of the candidate idea database DB04.
  • a blank or a symbol "-" indicating a blank is entered for an experiment that has not been presented in the past.
  • the idea expression database DB01 stores both previously proposed materials (experimental results) and unproposed materials (experimental results).
  • the Time Stamp column T1C2 represents the date of the experiment.
  • the mwt column T1C3 indicates the molecular weight. In this example, the molecular weight is one of the target variables to be improved, and the larger the value, the more desirable the amount.
  • the logP column T1C4 shows the water partition coefficient. In this example, logP is one of the objective variables to be improved, and the smaller the value, the better.
  • the SMILES column T1C5 shows a chemical structural formula represented by the SMILES notation.
  • the catalyst column T1C6 shows the catalyst, one of the experimental conditions.
  • the temp column T1C7 indicates the synthesis temperature, which is one of the experimental conditions.
  • the idea expression database DB01 includes columns for some estimated values of compound properties. As will be described later, the VAE of the idea reduction module P02 receives property values of compounds as inputs and outputs predicted values of those property values.
  • pred_mwt column T1C8 indicates the predicted mwt value of candidate ideas proposed in the past by this system.
  • pred_logP is the predicted value of logP of candidate ideas proposed in the past by this system.
  • pred_catalyst column T1C10 shows predicted values of catalysts proposed as candidate ideas in the past by the present system.
  • pred_temp column T1C11 indicates the predicted value of the temp of candidate ideas proposed in the past by the present system. In columns T1C8 to T1C11 containing pred_, blanks or a symbol "-" indicating blanks are entered for experiments that have not been presented in the past.
  • One or more KPIs can be set. By setting multiple KPIs, it is possible to propose candidate ideas that meet those KPIs.
  • the idea vectorization module P01 generates an idea vector database DB02 from the idea expression database DB01.
  • FIG. 5 shows a configuration example of the idea vector database DB02.
  • the idea vector database DB02 includes an idea_vector column T2C5 in addition to the data of the idea expression database DB01.
  • the idea_vector column T2C5 indicates a vector of idea expressions, in this example, a vector of chemical structural formulas expressed in SMILES notation.
  • FIG. 6 is a flowchart of a processing example of the idea vectorization module P01.
  • the idea vectorization module P01 reads parameter data from the parameter database DB06. Subsequently, the idea expression data is read from the idea expression database DB01 (S101).
  • the idea vectorization module P01 uses the vocabulary information described in the parameter data to convert the chemical structural formula expressed in SMILES notation into a vector (matrix).
  • the matrix and the vector are examples of the method of representing the array, and the same array can be represented by the vector and the matrix.
  • this conversion method for example, a method widely known as one-hot encoding can be used.
  • chemical structural formulas and vectors have one-to-one correspondence and can be reversely transformed.
  • the idea vectorization module P01 duplicates the idea expression database DB01, adds an idea_vector column T2C5, stores the corresponding vector after conversion, and writes it as the idea vector database DB02.
  • FIG. 7 shows an example of generating a structural formula vector, which is an idea vector, from a chemical structural formula, which is an idea expression.
  • the idea vectorization module P01 converts a chemical structural formula represented by the SMILES notation into a vector by one-hot encoding.
  • the idea contraction module P02 generates a contraction vector database DB03 from the idea vector database DB02.
  • FIG. 8 shows a configuration example of the contracted vector database DB03.
  • the contracted vector database DB03 includes an abst_vector column T3C5 in addition to the data of the idea vector database DB02.
  • the abst_vector column T3C5 stores contraction vectors of idea vectors.
  • the idea vector is a vector representing a chemical structural formula expressed in SMILES notation.
  • FIG. 9 is a flowchart showing a processing example of the idea reduction module P02.
  • the idea reduction module P02 first reads parameter data from the parameter database DB06 (S151). Further, the idea reduction module P02 reads model data from the model database DB05 (S152). Furthermore, the idea contraction module P02 reads idea vector data from the idea vector database DB02 (S153).
  • the idea reduction module P02 instantiates the VAE model (S154). If the trained model is saved, instantiate the trained model. If the learned VAE is not stored, the idea reduction module P02 initializes the (neural network) VAE using initial values described in the parameter data.
  • the VAE encoder can be composed of, for example, an outer encoder 211 and an inner encoder 212 in the subsequent stage.
  • the outer encoder 211 may be composed of multiple one-dimensional convolutional layers and the inner encoder 212 may be composed of multiple fully connected layers.
  • the encoder receives, as input data 201, a vector 202 of chemical structural formulas that can be represented by an M ⁇ N-dimensional matrix and a vector 203 in which explanatory variables other than SMILES are concatenated.
  • the VAE encoder converts input data 201 into an L ⁇ 1-dimensional vector 214 (hereinafter simply referred to as an L-dimensional vector). This vector 214 is a contracted vector in the latent space 213 .
  • the encoder connects the output vector of the final layer of the outer encoder 211 (convolutional layer) and the vector obtained by connecting explanatory variables other than SMILES included in the idea vector database DB02 to generate a vector V1.
  • the encoder inputs vector V 1 to an inner encoder 212 (fully connected layer) following the final layer of convolutional layers to produce an L-dimensional vector 214 .
  • the KPIs mwt and logP and the experimental conditions catalyst and temp are contained in vector 203 (vector V1) and input to the inner encoder 212 of the encoder. By doing so, it is possible to encode including characteristic values and experimental conditions.
  • the input data does not need to include explanatory variables other than SMILES if the estimated values of the properties of the compound are unnecessary.
  • the decoder can be composed of, for example, an inner decoder 215 and an outer decoder 216 in the subsequent stage.
  • the inner decoder 215 can be composed of multiple fully-connected layers, and the outer decoder 216 can be composed of an RNN (Recurrent Neural Network).
  • the decoder receives the L-dimensional vector 214 as an input and inversely transforms it into a vector 222 representing chemical structural formulas corresponding to the M ⁇ N-dimensional matrix and a vector 223 consisting of explanatory variables other than the chemical structural formulas.
  • the output chemical structure 222 may differ slightly from the input chemical structure.
  • the output data 221 includes predicted values 223 of explanatory variables other than the input chemical structural formula.
  • the encoder Since the encoder also encodes explanatory variables other than the matrix obtained by transforming SMILES, the output vector V2 of the final layer of the fully connected layer is divided. This also yields (predicted values of) recovered explanatory variables such as mwt, logP, catalyst, and temp. The bonding position of V1 and the splitting position of V2 match.
  • FIG. 10 shows an example of a model for forming a contracted vector by the idea contraction module P02, and any model can be used according to the design.
  • steps after step S155 are followed by recursive processing (learning processing) based on batch processing generally used in deep learning. That is, out of the learning data, the number of data specified by the batch size is processed. The time until all batches are processed once is called one epoch, and actual learning is repeatedly executed for multiple epochs. Processing performed in one batch will be described below.
  • the idea contraction module P02 inputs idea vectors to be learned to the VAE for the number of batches (S155).
  • a reconstruction error U0 is calculated from the error between the input of the encoder and the output of the decoder (S156).
  • the reconstruction error U0 may be calculated by any method, for example, it may be the average value of the VAE input and output errors for each sample included in the batch.
  • the reconstruction error is calculated including those parts.
  • the idea reduction module P02 calculates a correlation coefficient between a predetermined component in the idea reduction vector and the target variable (KPI) (S157). For example, the idea reduction module P02 calculates a correlation coefficient R0 between a predetermined first component of the encoder output vector (contracted vector) and mwt. Next, a correlation coefficient R1 between a predetermined second component of the encoder output vector (contracted vector) and logP is calculated.
  • CorrDimIndex which objective variable and which component are to be correlated is specified by "CorrDimIndex" in the parameter data.
  • the correlation calculation method is arbitrary. For example, you may define a kernel function to compute the correlation, you may use the correlation coefficient computed based on the covariance and the mean, or you may use the sum of the correlation coefficients and weights computed in different ways. may
  • the idea reduction module P02 calculates the error (difference) between the true value of the characteristic value and the predicted value.
  • the true value is the input to the VAE and the predicted value is the output.
  • the idea contraction module P02 calculates the difference between the value of the mwt column T1C3 and the value of the pred_mwt column T1C8 of the idea vector database DB02, and the difference between the value of the logP column T1C4 and the value of the pred_logP column T1C9, and sums them W0 is calculated (S158).
  • the sum W0 is, for example, the sum of differences in characteristic values of a plurality of samples.
  • the sum of differences of different characteristic values may be a simple sum or a sum after being multiplied by a weight designated by parameter data.
  • calculation of the error W0 of the characteristic value (mwt, logP) may be omitted.
  • VAE input and output data may not include characteristic values (KPIs).
  • Experimental conditions can be omitted as errors are not included in the Loss function.
  • the idea reduction module P02 calculates the loss from the sum of the reconstruction error U0, the correlation coefficient R0, the correlation coefficient R1, and the difference W0 between the true value and the predicted value of the objective variable. Since the neural network learns so that the loss becomes small, the code is appropriately set. For example, the codes of R0 and R1 are set in advance by "CorrDir" of the parameter data.
  • the idea reduction module P02 updates the VAE so that the parameters are optimized based on this Loss (S159).
  • the VAE parameters are updated such that the reconstruction error U0 and the characteristic value error W0 are reduced and the correlations R1 and R2 are increased.
  • the sign of the variable in the Loss function is determined by design. For example, the sign may be determined so that the Loss function becomes smaller when the correlation becomes smaller.
  • the idea reduction module P02 determines whether learning has ended (S160). In this example, the end flag is set when the learning process is completed for a predetermined number of epochs. Otherwise (S160: NO), the process returns to step S155 and continues.
  • the idea reduction module P02 writes and saves the parameters of the learned model in the model database DB05 (S161).
  • the idea contraction module P02 receives all idea vectors once again from the learned VAE, acquires the output of the encoder, and converts them into contracted vectors (S162).
  • the idea contraction module P02 adds an abst_vector column T3C5 to the configuration of the idea vector database DB02 to store contracted vectors, and writes this table as contracted vector database DB03 (S163).
  • FIG. 11 is a flow chart showing a processing example of the contracted space search module P03.
  • the contracted space search module P03 first reads parameter data from the parameter database DB06 (S201). Subsequently, the contracted space search module P03 reads the contracted vector database DB03 (S202).
  • the contracted space search module P03 extracts a contracted vector that serves as the search center from the contracted vector database DB03 based on the search conditions described in the parameter data (S203).
  • the target value of each objective variable is described in "TargetValue" of section 62 of the parameter data.
  • samples (records) with mwt close to 1.6 and logP close to -1.5 are extracted from the contracted vector database DB03.
  • a predetermined range indicating how close to extract is specified by "FilteringRange" of the parameter data.
  • the number to be extracted is arbitrary and may be specified by parameter data. Note that records in which at least one of mwt and logP is close to the target value may be extracted, or a reference different from the target value, for example, a record specified by the user may be extracted.
  • the contracted space search module P03 generates a new contracted vector by adding a small real number as a predetermined bias to the first and second components of the extracted contracted vector (S204).
  • the target variable values of the new reduced vector are slightly different from those of the original reduced vector.
  • the number of bias sets consisting of one or more component biases to give each contraction vector may be one or two or more. A different set of biases can be applied to each contracted vector to generate multiple new contracted vectors from each contracted vector.
  • the bias applied to each component may be a preset value, and one or a different bias can be applied to each extracted contracted vector.
  • the applied bias set may be common or different for different components.
  • the extracted contracted vectors and the contracted vectors newly generated by the bias are candidates for compounds (ideas) to be presented to the user.
  • the contracted space search module P03 extracts all or part of the records in which valid values are stored in the SuggestID column T1C21 of the contracted vector database DB03. These are records that have been proposed in the past.
  • the reduced space search module P03 calculates the degree of similarity between each of the extracted or generated contracted vectors and the contracted vectors of the extracted proposed records.
  • the degree of similarity is represented by Euclidean distance, for example.
  • component 0 and component 1 values are excluded from the similarity calculation. These values may be included in similarity calculations.
  • the contracted space search module P03 uses the calculated similarity to filter the contracted vector extracted based on the target value and the newly generated contracted vector (S205).
  • the novelty of candidate ideas is emphasized, and filtering is performed so that contracted vectors that are separated from all proposed ideas by a predetermined threshold or more remain.
  • the threshold is specified by "threshold" of the parameter data.
  • the contracted space search module P03 refers to the TimeStamp column T1C2 of the contracted vector database DB03, and extracts records within a certain period from the current date and time.
  • the contracted space search module P03 calculates the degree of similarity between the contracted vector included in each extracted record and each candidate contracted vector selected by the above filtering.
  • the degree of similarity can be represented by Euclidean distance, for example, as described above. Similarity calculations may be the same or different between submitted ideas and recent ideas.
  • the contracted space search module P03 selects a portion from the candidate contracted vectors by filtering using the degree of similarity (S206).
  • filtering is performed so that candidate contraction vectors that are more than a threshold apart from all records within a certain period of time from the current date and time remain.
  • the threshold can be specified by "threshold_Time" of the parameter data, and is set to 30 days in this example. Note that filtering may be performed on only one of the degree of similarity to ideas that have already been presented or the degree of similarity to recent ideas, or these filtering may be performed independently.
  • the contracted space search module P03 passes the candidate contracted vectors remaining after filtering to the candidate idea generation module P04 (S207).
  • the candidate idea generation module P04 generates candidate ideas and stores them in the candidate idea database DB04.
  • FIG. 12 shows a configuration example of the candidate idea database DB04.
  • the SuggestID column T4C1 indicates the ID of the proposed idea.
  • the SuggestDate column T4C2 represents the proposed date.
  • the pred_mwt column T4C3 indicates the predicted value of mwt of the candidate idea.
  • the pred_logP column T4C4 column indicates the predicted value of logP of candidate ideas proposed in the past by this system.
  • the SMILES column T4C5 shows the SMILES notation of candidate compounds (candidate ideas) that this system has proposed as candidate ideas in the past.
  • the pred_catalyst column T4C6 indicates predicted values of catalysts proposed as candidate ideas in the past by the present system.
  • the pred_temp column T4C7 indicates predicted values of temp of candidate ideas proposed in the past by the present system.
  • Columns T41C to T4C2 are associated with columns of the same name in the idea expression database DB01, and the same proposal has the same value.
  • FIG. 13 is a flowchart of a processing example of the candidate idea generation module P04.
  • the candidate idea generation module P04 first reads parameter data (S251), and then receives a contracted vector, which is an abstract representation of the candidate idea, from the contracted space search module P03 (S252).
  • the candidate idea generation module P04 reads the learned model from the model database DB05 (S253).
  • the model read here is the same as the VAE used in the idea reduction module P02.
  • the candidate idea generation module P04 inputs the contracted vector from the contracted space search module P03 to the decoder of the learned VAE to obtain a matrix that is the idea vector (S254).
  • the idea reduction module P02 encodes the explanatory variables mwt and logP and the experimental conditions catalyst and temp, which are explanatory variables other than SMILES. Therefore, the candidate idea generation module P04 divides the intermediate output vector V2 to obtain mwt and logP and catalyst and temp outputs. This process is not performed if only SMILES are encoded.
  • the candidate idea generation module P04 inversely transforms the inversely transformed idea vector into an idea expression according to a predetermined method (S255).
  • a chemical structural formula in SMILES notation which is an idea expression, is obtained by one-hot decoding the decoded matrix.
  • the candidate idea generation module P04 puts together the decrypted SMILES, characteristic values, and experimental conditions, and stores them as new records in the candidate idea database DB04. At this time, a unique identifier is added to the SuggestID column 4T1C, and an execution date and time is added to the SuggestDate column T4C2.
  • the candidate idea generation module P04 displays the updated data added to the candidate idea database DB04 via the display unit P07 of the user interface M05 to present candidate ideas to the user.
  • Example 2 An example of a candidate idea generation system based on document analysis technology will be disclosed as Example 2.
  • Various documents such as patent documents, academic papers, and in-house reports, as well as sentences that are constituent elements in each document, are also forms of expression of ideas. According to this system, it is possible to generate and present new candidate ideas based on such known ideas, thereby supporting the user's idea generation.
  • FIG. 14A and 14B show examples of the user interface (screen) of the idea support application according to the second embodiment. After the user operation on the screen H10A shown in FIG. 14A, the user operation on the screen H10B shown in FIG. 14B is executed.
  • the object H13 shows the keyword "your hand”.
  • four types of keywords are arranged as word cards. These are the current dominant set of words for describing your ideas.
  • the system presents new words (word cards) in the "new word” object H12.
  • word cards new words
  • the user evaluates whether or not the current card in hand is sufficient for verbalizing the idea, and inputs it into the system by selecting the button indicated by the "explanation level of current card in hand” H11, for example, on a 5-point scale.
  • the user compares the words on hand with the new words, selects the word card that is most unrelated to his/her idea, and presses the "Next" button H14 to discard the selected word card. .
  • the system displays the next screen H10B shown in FIG. 14B.
  • the system displays the new word card in the "new word” object H12 on the screen H10B shown in FIG. 14B.
  • Object H13 indicates the keyword "your hand”. Similar to the operation described with reference to FIG. 14A, the user evaluates whether or not the current card in hand is sufficient for verbalizing the idea. Enter the system by selecting a button. The user compares the words on hand with the new words, selects the word card that is most irrelevant to his idea, and presses the "Next" button H14 to execute the operation of discarding the selected word card. After replacing the word cards as necessary, the system displays the next screen (not shown).
  • FIG. 15 shows a configuration example of the evaluation result table T60.
  • the step column T6C1 indicates identifiers of user evaluation steps.
  • the word column T6C2 shows the hand-held word group for each evaluation step.
  • a vector representing this group of words is an example of a word vector. This word vector indicates the presence or absence of words included in the hand-held words in the evaluation step with 0s and 1s.
  • the evaluation index column T6C3 indicates the evaluation result of the evaluation step.
  • the evaluation index column T6C3 is composed of a differential evaluation column T6C5 and an absolute evaluation column T6C6.
  • the user evaluation of each evaluation step input on the evaluation screen is stored in the absolute evaluation column T6C6.
  • the difference between the absolute evaluation of the word vector in the previous step and the absolute evaluation of the next word vector is stored in the difference evaluation field T6C5.
  • the user can keep word cards suitable for explaining his/her own idea, and proceed with the verbalization of the idea. can.
  • the idea vectorization module P01 in the second embodiment receives an idea expression in which a single sentence or a plurality of sentences is one unit, and converts it into an idea vector, which is a real number vector.
  • an idea vector which is a real number vector.
  • Word2Vec and Doc2Vec are widely known methods for converting such natural language expressions into real number vectors.
  • FIG. 16 shows an example of word vector T70.
  • a document to be converted is one record (one word vector), and one word vector indicates the presence/absence of words contained in the document with 0s and 1s.
  • the system searches for the closest group of words, relying only on the degree of similarity (distance) between word vectors, and determines the following new words.
  • the three records in FIG. 16 are a group of word vectors having different values only for one component.
  • the system searches for similar word vectors and extracts the second and third records.
  • the system randomly selects the third record among them, and presents the difference "resolution" to the next new word object H12.
  • the evaluation result table T60 in FIG. 15 is an example of the idea vector database DB02.
  • the idea reduction module P02 of the second embodiment may use VAEs as in the first embodiment.
  • the idea reduction module P02 takes a word vector as input and trains the network using a loss function that strengthens the correlation between the predetermined component of the reduced vector obtained by the encoder and the user evaluation value.
  • the contracted space search module P03 can search for useful word vectors that contribute to organizing ideas. Since new word vectors can be obtained by the candidate idea generation module, it is possible to support thinking according to the user's habits and interests.
  • any index value related to the quality of the idea such as the evaluation value of the document that was the basis for generating the word vector, or the evaluation from other users, can be used. do not have.
  • the present invention is not limited to the above-described embodiments, and includes various modifications.
  • the above-described embodiments have been described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the described configurations.
  • it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment and it is also possible to add the configuration of another embodiment to the configuration of one embodiment.
  • each of the above configurations, functions, processing units, etc. may be realized by hardware, for example, by designing a part or all of them with an integrated circuit.
  • each of the above configurations, functions, etc. may be realized by software by a processor interpreting and executing a program for realizing each function.
  • Information such as programs, tables, and files that implement each function can be stored in a memory, a hard disk, a recording device such as an SSD (Solid State Drive), or a recording medium such as an IC card or SD card.
  • control lines and information lines indicate what is considered necessary for explanation, and not all control lines and information lines are necessarily indicated on the product. In fact, it may be considered that almost all configurations are interconnected.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Computational Mathematics (AREA)
  • Algebra (AREA)
  • Databases & Information Systems (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
PCT/JP2022/008022 2021-03-19 2022-02-25 候補アイディアを生成するシステム及び方法 Ceased WO2022196301A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP22771057.1A EP4310740A4 (en) 2021-03-19 2022-02-25 SYSTEM AND METHOD FOR GENERATING CANDIDATE IDEAS
US18/272,233 US12475195B2 (en) 2021-03-19 2022-02-25 System and method for generating candidate idea

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021-045937 2021-03-19
JP2021045937A JP7594951B2 (ja) 2021-03-19 2021-03-19 候補アイディアを生成するシステム

Publications (1)

Publication Number Publication Date
WO2022196301A1 true WO2022196301A1 (ja) 2022-09-22

Family

ID=83321229

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/008022 Ceased WO2022196301A1 (ja) 2021-03-19 2022-02-25 候補アイディアを生成するシステム及び方法

Country Status (4)

Country Link
US (1) US12475195B2 (cg-RX-API-DMAC7.html)
EP (1) EP4310740A4 (cg-RX-API-DMAC7.html)
JP (1) JP7594951B2 (cg-RX-API-DMAC7.html)
WO (1) WO2022196301A1 (cg-RX-API-DMAC7.html)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12682357B2 (en) * 2022-05-25 2026-07-14 Arizona Board Of Regents On Behalf Of Arizona State University Framework for cyber-physical interaction aware test case generation to identify operational changes
US20250335750A1 (en) * 2024-04-30 2025-10-30 Toyota Research Institute, Inc. Systems and methods for using a generative model to generate an output based on a set of diverse inputs

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2020095452A (ja) 2018-12-12 2020-06-18 株式会社日立製作所 語彙抽出支援システムおよび語彙抽出支援方法
JP2020187642A (ja) * 2019-05-16 2020-11-19 富士通株式会社 最適化装置、最適化システム、最適化方法および最適化プログラム

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005063312A (ja) * 2003-08-19 2005-03-10 Toshiba Corp 発想支援装置、発想支援方法および発想支援プログラム
US7966327B2 (en) * 2004-11-08 2011-06-21 The Trustees Of Princeton University Similarity search system with compact data structures
US8396870B2 (en) * 2009-06-25 2013-03-12 University Of Tennessee Research Foundation Method and apparatus for predicting object properties and events using similarity-based information retrieval and modeling
CN110023924A (zh) * 2016-11-11 2019-07-16 德尼梅尔奥克提麦恩有限公司 用于语义搜索的设备和方法
US10606885B2 (en) * 2016-11-15 2020-03-31 Evolv Technology Solutions, Inc. Data object creation and recommendation using machine learning based online evolution
US11537623B2 (en) * 2017-05-18 2022-12-27 Meta Platforms, Inc. Deep semantic content selection
US10824661B1 (en) * 2018-04-30 2020-11-03 Intuit Inc. Mapping of topics within a domain based on terms associated with the topics
US11971963B2 (en) * 2018-05-30 2024-04-30 Quantum-Si Incorporated Methods and apparatus for multi-modal prediction using a trained statistical model
JP7213754B2 (ja) * 2019-05-27 2023-01-27 株式会社日立製作所 情報処理システム、推論方法、攻撃検知方法、推論実行プログラム及び攻撃検知プログラム
EP3751467B1 (en) * 2019-06-14 2026-05-27 Robert Bosch GmbH A machine learning system
US11783518B2 (en) * 2019-09-30 2023-10-10 Lawrence Livermore National Security, Llc Few-view computed tomography reconstruction using deep neural network inference
EP4038618A1 (en) * 2019-10-25 2022-08-10 Syntensor Inc. Target-to-catalyst translation networks
US11379718B2 (en) * 2019-12-10 2022-07-05 International Business Machines Corporation Ground truth quality for machine learning models
WO2021183408A1 (en) * 2020-03-09 2021-09-16 Pioneer Hi-Bred International, Inc. Multi-modal methods and systems
US11409752B1 (en) * 2020-11-04 2022-08-09 Casetext, Inc. Dimensional reduction of complex vectors in artificially intelligent solutions to compare similarity of natural language text
US20220245440A1 (en) * 2021-01-29 2022-08-04 International Business Machines Corporation Triplet generation for representation learning in time series using distance based similarity search

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2020095452A (ja) 2018-12-12 2020-06-18 株式会社日立製作所 語彙抽出支援システムおよび語彙抽出支援方法
JP2020187642A (ja) * 2019-05-16 2020-11-19 富士通株式会社 最適化装置、最適化システム、最適化方法および最適化プログラム

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
RAFAEL GóMEZ-BOMBARELLI, JENNIFER N. WEI, DAVID DUVENAUD, JOSé MIGUEL HERNáNDEZ-LOBATO, BENJAMíN SáNCHEZ-: "Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules", ACS CENTRAL SCIENCE, vol. 4, no. 2, 28 February 2018 (2018-02-28), pages 268 - 276, XP055471225, ISSN: 2374-7943, DOI: 10.1021/acscentsci.7b00572 *
See also references of EP4310740A4

Also Published As

Publication number Publication date
US12475195B2 (en) 2025-11-18
EP4310740A4 (en) 2025-03-05
US20240078290A1 (en) 2024-03-07
JP7594951B2 (ja) 2024-12-05
EP4310740A1 (en) 2024-01-24
JP2022144778A (ja) 2022-10-03

Similar Documents

Publication Publication Date Title
CN116719520B (zh) 代码生成方法及装置
EP4248309B1 (en) Automated merge conflict resolution with transformers
CN118093834B (zh) 一种基于aigc大模型的语言处理问答系统及方法
US20240362209A1 (en) Systems and methods for automatically generating source code
CN117312989B (zh) 一种基于GCN和RoBERTa的上下文感知的列语义识别方法和系统
WO2022203829A1 (en) Semi-supervised translation of source code programs using neural transformers
WO2018131259A1 (ja) 文章評価装置、及び文章評価方法
CN115048447B (zh) 一种基于智能语义补全的数据库自然语言接口系统
US12259864B1 (en) Apparatus and method for training a machine learning model
WO2017006104A1 (en) Improved artificial neural network for language modelling and prediction
US20240087683A1 (en) Classification using a machine learning model trained with triplet loss
CN116861269A (zh) 工程领域的多源异构数据融合及分析方法
Bazaga et al. Translating synthetic natural language to database queries with a polyglot deep learning framework
US12475195B2 (en) System and method for generating candidate idea
CN120067464A (zh) 一种基于大语言模型的跨模态协同推荐方法及系统
Luo et al. HierarT: Multi-hop temporal knowledge graph forecasting with hierarchical reinforcement learning
Abimbola et al. Open-source embedding models: A comprehensive survey of techniques, benchmarks, and applications
Alagarsamy et al. An experimental analysis of optimal hybrid word embedding methods for text classification using a movie review dataset
CN119903190B (zh) 一种结合生成式语言模型与语义文档图谱的多文档问答检索方法
US20260099968A1 (en) System and methods for generating composite images
CN117573096B (zh) 一种融合抽象语法树结构信息的智能代码补全方法
CN117555950A (zh) 基于数据中台的数据血缘关系构建方法
Ranzato A text segmentation technique based on language models
CN120636783B (zh) 一种基于自然语言交互的ai诊断模型动态调优方法及系统
Pan et al. Cross-modal de-deviation for enhancing few-shot classification

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22771057

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18272233

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 2022771057

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2022771057

Country of ref document: EP

Effective date: 20231019

WWG Wipo information: grant in national office

Ref document number: 18272233

Country of ref document: US