JP2024041902A - Multi-source interoperability and/or information retrieval optimization - Google Patents

Abstract

A method and system for reducing query-related resource usage in a data retrieval process is provided. The method takes a graph query associated with a data request and converts the graph query into a query set based on a graph data model and a pattern of the graph query. The generated query set may include queries and query operators linking the queries, the query operators including a first query operator linking a first query and a second query. Prior to executing the first query or the second query, predicting a satisfiability problem, and based on the prediction, removing the first query operator from the query set to update the query set, and executing the updated query set to satisfy the fluff query.

Description

(Cross reference to related applications)
This application claims the benefit of the following United States applications filed June 13, 2018, the entire contents of which are incorporated herein by reference: (1) U.S. Patent Application No. 16/007,639, Title: "System and Method for Reducing Data Retrieval Delay Through Prediction-Based Data Subgraph Generation," (2) U.S. Patent Application No. 16/007, No. 911, entitled ``Systems and Methods for Providing Generation of Graph Data Models Based on Predictive Models.'' ``Systems and Methods for Reducing Resource Utilization.''

The present invention relates to facilitating multi-source interoperability and/or information retrieval optimization.

Typically, as an organization grows, the number of data silos that exist within the organization increases, creating many repositories of fixed data that are not "connected" to other departments in the organization. Such data silos often exist because the data formats or data source technologies do not match those previously or currently in standard use. For example, large enterprises may have multiple legacy data systems inherited that may not be compatible with each other or with new data systems. Although it is possible to use data migration systems to transfer data from various data storage types, formats or IT systems to one data system that supports current standards, such large-scale data migrations The process typically requires significant overhead (computing resources, time, etc.) and can cause significant disruption to other organizational activities. Problems such as those mentioned above and others exist.

Aspects of the invention relate to methods, apparatus, or systems for facilitating multi-source interoperability or information retrieval optimization, e.g., multi-source type queries against multiple data sources of different data source types. using a predictive model to generate a data transformation model; using a predictive model to anticipate requests for transformed data and store such data in temporary data storage; This field relates to the optimization of query sets derived from data requests in order to reduce resource usage related to queries in the data search process.

Certain embodiments reduce query-related resource usage in the data retrieval process. As an example, a graph query related to a data request may be obtained. A graph query may be converted into a query set based on the graph data model and the pattern of the graph query. The generated query set may include a query and a query operator that links the queries, the query operator being a first query operator or other query operator that links a first query and a second query of said queries. Contains operators. A satisfiability problem may be predicted before the first query and the second query are executed, and the satisfiability problem is determined from the first query and the second query. Concerning combining derived results. Based on the prediction, the first query operator is removed from the query set and the query set is updated. The updated query set may be executed to respond to the graph query.

Various aspects, features and advantages of the invention other than those described above will become apparent from reference to the detailed description of the invention and the accompanying drawings. Furthermore, it is to be understood that both the above summary and the following detailed description are illustrative only and are not intended to limit the scope of the invention. As used in this specification and the claims, the singular forms "a," "an," and "the" refer to the plural. However, this does not apply if it is clear that it should be interpreted differently depending on the context. Additionally, as used herein and in the claims, the term "or" means "and/or." However, this does not apply if it is clear that it should be interpreted differently depending on the context.

1 illustrates a system for facilitating multi-source interoperability and information retrieval optimization in accordance with one or more embodiments.

1 illustrates an enterprise environment including one or more components of a system in accordance with one or more embodiments.

1 illustrates an example architecture of one or more components of a system, according to one or more embodiments.

4 illustrates an example graph in a graph database in accordance with one or more embodiments.

A data representation that is incompatible with a graph database, a template for converting an incompatible data representation to a graph data representation that is compatible with a graph database, and an incompatible data representation according to one or more embodiments. Figure 3 shows a graph data representation derived from the data representation.

FIG. 6 illustrates an example of two different sets of graph data representations generated based on templates from trained predictive models in accordance with one or more embodiments.

4 illustrates an example graph data model in accordance with one or more embodiments.

4 illustrates an example of predictive model inference in accordance with one or more embodiments.

1 illustrates an example of similarity prediction by a predictive model according to one or more embodiments.

3 illustrates example code for determining similarity between nodes of a graph in accordance with one or more embodiments.

4 illustrates an example of generating new nodes or edges and learning rules and ontology matches in accordance with one or more embodiments; FIG.

1 illustrates a method for providing generation of a graph data model based on a predictive model in accordance with one or more embodiments.

4 illustrates a method for reducing data retrieval delay through prediction-based generation of data subgraphs in accordance with one or more embodiments.

1 illustrates a flowchart of a method for reducing query-related resource usage in a data retrieval process according to one or more embodiments.

In the following description, numerous specific details are set forth for purposes of explanation in order to provide a thorough understanding of embodiments of the invention. However, one of ordinary skill in the art will understand that embodiments of the invention may be practiced without these specific details or with equivalent construction. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring embodiments of the present invention.

FIG. 1 illustrates a system 100 for facilitating multi-source interoperability and optimization of information retrieval, according to one or more embodiments. As illustrated in FIG. 1, the system 100 may include a server 102, client devices 104a-104n, a data source 132, and other components. The server 102 may include a data management subsystem 112, a model management subsystem 114, a request subsystem 116, an optimization subsystem 118, a presentation subsystem 120, an electronic storage device 122, and other components. Each of the client devices 104 may include any type of mobile terminal, fixed terminal, or other device. The client devices 104 may be, by way of example, a desktop computer, a notebook computer, a tablet computer, a smartphone, a wearable device, or other client device. A user may, for example, utilize one or more client devices 104 to interact with one or more servers 102 or other components of the system 100. The data sources 132 may include graph data sources 134, unstructured data sources (e.g., typewriter documents), semi-structured data sources (e.g., XML documents, emails, etc.), or structured data sources (e.g., Relational Database Management System (RDBMS) information, Lightweight Directory Access Protocol (LDAP) information, etc.) 136, and other data sources 138. The data sources may include various databases or other data sources. In some embodiments, a database may include one or more data sources 132. It should be noted that although one or more processes are described herein as being performed by particular components of the server 102, in some embodiments, these processes may be performed by other components of the server 102 or other components of the system 100. As an example, one or more processes described herein as being performed by components of the server 102 may be performed by components of the client device 104 in some embodiments.

In some embodiments, enterprise environment 200 may be an environment for one or more components of system 100, as shown in FIG. Enterprise environment 200 may include an application and analytics portion 202, an enterprise “data lake” portion 204, a virtual portion 206, and other portions. The server 102 provides reports 208, analysis 210, machine learning or data mining information 212, unstructured information 214 (e.g., typewritten documents), semi-structured information 216 (e.g., XML documents, emails, etc.), structured 218 (e.g., RDBMS information, LDAP information, etc.) and other information. Server 102 may receive such information, for example, from one or more client devices 104 executing various applications or analysis operations or from other sources.

In some embodiments, architecture 300 may be an environment for one or more components of system 100, as shown in FIG. As shown in FIG. 3, the server 102 includes a metadata extractor 302, a text extractor 304, a graph extractor 306 (e.g., a Resource Description Framework (RDF) extractor), a machine learning component 308, and a geospatial index 310. , a graph index 312, a text index 314, a relational mapping component 316, a query engine 318, and other components. In some embodiments, one or more of these components are included in or perform operations associated with one or more of the server components shown in FIG. Good too. For example, metadata extractor 302, text extractor 304, graph extractor 306, or other components may be included in data management subsystem 112 shown in FIG. Machine learning component 308 may be or be included in model management subsystem 114 shown in FIG. Geospatial index 310, graph index 312, text index 314, relational mapping component 316, query engine 318, or other components may be part of model management subsystem 114, request subsystem 116, or optimization subsystem 118 shown in FIG. or may be included therein. In some embodiments, server 102 is associated with document storage system 320.

In some embodiments, as shown in FIG. 3, server 102 may receive graph data, unstructured data, information from a relational database management system, or data from other sources 134. . As described herein, a component of server 102 (eg, query engine 318) may execute one or more queries to obtain related results. Based on templates or other information generated by the system (e.g., geospatial index 310, graph index 312, text index 314, information from relational mapping component 316), as described herein, Server 102 transforms the data representation obtained from the query into a graph format (or other format) via one or more data transformation models.

In some embodiments, system 100 uses a data transformation model or other device configured to convert a data representation of one data source to a data representation compatible with another data source (or vice versa). Facilitate multi-source interoperability between different data source technologies or standards through data model generation. In some embodiments, the system 100 utilizes such data transformation models to transform incompatible query results (e.g., between different data source types) into results that are compatible with the desired data source. can facilitate multi-source queries to multiple data sources of different data source types. In this manner, for example, the system 100 can save a company or other entity the hassle of having to review legacy or current databases in order to adopt new or different data source technologies or standards. . As one example of use, system 100 can transform data representations from one or more data sources of various data source types on the fly via one or more such data transformation models.

In some embodiments, the system 100 may obtain one or more templates for converting a data representation of a first data source type (e.g., a relational model type or other data source type) to a data representation of a second data source type (e.g., a graphic model type or other data source type) and may create or modify a data conversion model based on the obtained templates. As an example, the template may include instructions for converting data characteristics corresponding to the first data source type (e.g., row or column attributes and values specific to a particular SQL data source or other data source) to data characteristics corresponding to the second data source (e.g., graph attributes and values specific to a graph data source or other data source). As a further example, the system 100 may process the template to determine a pattern (e.g., a regular expression or other pattern) or rule (associated with the template) for matching the data representation of the first data source type to at least one of the templates that can be used to convert a non-graph data representation to a data representation of the second data source type. The system 100 may then generate a data conversion model and incorporate the pattern, rule, or other modeling information as part of the data conversion model (e.g., such that the data conversion model includes or indicates such template, pattern, rule, etc.). In some embodiments, system 100 may utilize one or more predictive models (e.g., neural networks or other machine learning models) to generate one or more graph data models configured to transform a data representation (that is not compatible with a particular graph database) into a graph data representation that is compatible with a graph database, as described later herein, or generate one or more other data transformation models.

In some embodiments, system 100 may reduce delays to provide sufficient responses to requests or improve the efficiency of use of temporary data storage or other computer resources. System 100 can selectively retrieve and temporarily retrieve a subset of query results associated with a predicted request, e.g., by temporarily storing predictions of requests and query results associated with the predicted requests in a graph format. Delay reduction or efficiency improvements may be provided through storage, query set optimization, or other techniques. In one example, a request for query results is predicted, a subset of the results is retrieved in response to the request prediction, and a subset of the results is stored in a server cache, web cache, memory cache, or other temporary data storage (e.g., electronic storage 122). May be stored. A subset of the results may be converted into one or more subgraphs (eg, if the results are not in an appropriate graph format) and the subgraphs may be stored in temporary data storage. When a predicted request (or a future request that matches a predicted request) occurs, a conversion from a non-graph format to a graph format (for example, from a non-graph format to a graph format may be necessary in some cases, and other One or more subgraphs (instead of retrieving a subset of results via data storage) may be retrieved from temporary data storage and used to respond to predicted requests that occur. This way, for example, by temporarily storing results in a transformed format (before a particular request occurs), you can significantly reduce latency and other delays in fully responding to a request. It may be possible to reduce it.

In some embodiments, the system 100 improves performance in the data retrieval process by optimizing query sets derived from data requests, such as query sets that are converted into graph queries (or other queries related to the data request). Query-related resource usage can be reduced. In some embodiments, such query set optimizations include removing query operators that link multiple queries from the query set, merging multiple queries in the query set into a single query, or may include removing multiple queries from the query set or other optimizations. Such optimizations may be based on the prediction of one or more satisfiability (e.g., related to the combination of multiple results obtained from a particular query), incompatibility problems, or other problems. may be implemented to avoid or reduce such problems or the adverse effects of such problems. In some embodiments, in response to obtaining a graph query associated with a data request, system 100 combines the graph query with multiple queries and query operators (e.g., union, join) that link the queries. or other query operators). Based on the prediction of satisfiability problems (related to the combination of results obtained from the two queries), the system 100 either removes the query operator that links the two queries from the query set or Other optimizations may be performed on the set to update the query set. As described above, when the system 100 executes an updated query set to satisfy a graph query (and thus a data request), the system 100 may: i) execute one or more portions of the query set; (ii) delays resulting from such executions and attempts; or (iii) other adverse effects; Satisfiability problems or the negative effects of satisfiability problems can be avoided or reduced.

In some embodiments, system 100 performs predictive model-based (i) generation of data models (e.g., data transformation models, graph data models, etc.); (ii) generation of results related to future requests or other information; (iii) query generation or execution, (iv) query set optimization, or (v) other operations. Predictive models may include neural networks, other machine learning models, and other predictive models. As an example, a neural network may be based on a large number of neural units (or artificial neurons). A neural network may loosely mimic the workings of a biological brain (e.g., by large clusters of biological neurons connected by axons). Each neural unit of the neural network may be connected to many other neural units of the neural network. Such connections can have an enhancing or inhibiting influence on the activation state of the connected neural units. In some embodiments, each neural unit may have a summation function that combines the values of all inputs. In some embodiments, each connection (or neural unit itself) may have a threshold function that must be exceeded for the signal to propagate to other neural units. These neural network systems can be automatically learned and trained, rather than explicitly programmed, and may offer significantly better performance in solving problems in a given field than traditional computer programs. In some embodiments, a neural network may have multiple layers (eg, signal paths traverse from top to bottom layers). In some embodiments, the neural network may utilize backpropagation, in which forward stimuli are used to reset the weights to "upper layer" neural units. In some embodiments, stimulation and inhibition to a neural network may become more fluid as the interactions between connections become more chaotic and complex.

In some embodiments, the system 100 includes (i) modeling information associated with a data model (e.g., a template, its patterns or rules, or other information associated with a data transformation model, graph data model, etc.); ii) graph information related to nodes or edges of the graph or other information related to relationships of data points or other data sources; (iii) query information (e.g., graph queries, SQL or other non-graph queries; (iv) optimization information or problem information (e.g., information indicating queryset optimization logic or other optimizations related to the combination of results of a particular query); or other training information (eg, unstructured documents, semi-structured documents, structured documents, or other information). The system 100 trains one or more predictive models based on training information such as (i) one or more data models, graphs or other data structures, query sets or other information; or (ii) ) to generate one or more predictions associated with a predictive model, such as predictive patterns, rules, templates, optimization logic, satisfiability problems, attributes/values of graph nodes or edges, or other attributes/values. good. As an example, the number of information items (e.g., patterns, rules, templates, graphs, query sets or other training information items) used to train the predictive model may be 500 or more information items, 1000 or more information items , 10,000 or more information items, 100,000 or more information items, 1,000,000 or more information items, or any other number.

[Data model and data generation]

In some embodiments, model management subsystem 114 (i) generates one or more data models (e.g., data transformation models, graph data models, etc.) or information related thereto; (ii) generates patterns; It may be configured to predict rules, templates or other modeling information of the data model as described above, or (iii) perform other options. In some embodiments, the model management subsystem 114 provides data representations of a first data source type (e.g., a relational model type or other data source type) to a second data source type (e.g., a graphical model type). or other data source types), and a data transformation model may be created or modified based on the obtained templates. As an example, a template can match data characteristics (e.g., row or column attributes and values specific to a particular SQL data source or other data source) that correspond to a first data source type to a second data source. may include instructions for converting data characteristics (e.g., graph attributes and values specific to the graph data source or other data source) into data characteristics that are specific to the graph data source or other data source. As a further example, the model management subsystem processes at least one of the available templates to convert a data representation of a first data source type to a data representation of a second data source type. A pattern (eg, regular expression or other pattern) or rule (associated with a template) may be determined for matching a data representation of the first data source type to the first data source type. Model management subsystem 114 may then generate a data transformation model and incorporate patterns, rules, or other modeling information as part of the data transformation model.

As an example, as illustrated in FIG. It may be converted to a graph data representation that includes a data representation related to the video. As shown in FIG. 4, graph 400 includes various nodes 402 and edges 404 connecting nodes 402. In this example, two different movie nodes 402 corresponding to two different movies TT1583420 and TT016222 include a movie label node 402 (e.g. movie name 1 and movie name 2), director nodes NM000158 and NM000709, production company node C09940938 and It is connected to the publication date node 2000-12-22. These connections are illustrated by various edges 404 between individual nodes 402. Director nodes NM000158 and NM000709 and production company node C09940938 are also connected by edges 404 to corresponding label nodes 402 (eg, Person 1, Person 2, and Company Name).

As a further example, as shown in FIG. ) may be converted to As shown in FIG. 5, template 502 connects rows or columns (e.g., of table 500) to graph nodes or edges (e.g., of result graph 504) (or vice versa in some embodiments). ) may contain instructions for converting. In this example, table 500 includes information for two movies titled Movie 3 and Movie 4, such as identification code 506, title 507, year of release 508, box office country name 510, country code 512, and total revenue 514. Contains information. Template 502 (shown in graphical form in Figure 5) includes information on movie title, identification code, year of release, box office revenue, box office revenue country, box office revenue country label, box office revenue country code, and total revenue in that country. Contains a “placeholder” node 516. Multiple nodes 516 are linked by corresponding edges 518 that indicate relationships between these nodes 516. Based on the instructions of template 502 (eg, node 516), the non-graph data representation of table 500 for movie 3 is converted to the graph data representation of graph 504 (eg, node 520) that indicates node 520.

As shown in FIG. 1, data management subsystem 112 (i) generates one or more graphs or other data structures (e.g., SQL data structures, other non-graph data structures, etc.); and (ii) ) predict information for nodes, edges or other parts of such data structures; or (iii) perform other options. In some embodiments, data management subsystem 112 utilizes one or more graph data models to generate graphs from non-graph data representations (e.g., stored in SQL tables or other data sources). May be created or modified. As an example, data management subsystem 112 uses the graph data model to convert non-graph data representations (e.g., compatible with a particular graph database) to graph representations to create new graphs or Existing graphs in the database can be supplemented/modified.

In some embodiments, for one or more graphs, the request subsystem 116 performs the following operations (e.g., to determine all paths between two graph nodes; to determine the shortest path between two graph nodes; , to determine a predetermined number of shortest paths between two graph nodes, etc.) one or more path queries may be generated to find one or more paths between nodes of the graph. In some embodiments, the request subsystem 116 may generate a path query such that the path query includes restriction parameters for the path to be returned. As an example, one such path query may limit results to paths associated with transactions that exceed a specified amount (e.g., all nodes or edges of the shortest path have a value greater than or equal to $10,000). or must be associated with a transaction that exceeds another specified amount). As another example, a path query may limit results to paths associated with lifecycles (eg, product lifecycles, animal lifecycles, document lifecycles, etc.). As yet another example, a path query may limit results to paths from a movie node to nodes representing the production company and its parent company (e.g., the movie "Hitchcock" was created by 21st Century Fox, whose parent company is Fox It is produced by Fox Searchlight Pictures, which is owned by the studio. In some embodiments, request subsystem 116 may determine a query plan for responding to a data request based on such paths returned by such path queries. As an example, based on such path information, the request subsystem 116 may provide one for processing a data request (e.g., obtained from a user device or predicted as described herein). Alternatively, a plurality of queries may be determined, which graph or graph database will be the target source for processing the data request, and a cost associated with such target source. Request subsystem 116 may create or select a query plan for the data request based on such determination (e.g., by incorporating the target source into the query plan, determining the query or target source based on cost information, etc.). (determine priorities).

Some embodiments utilize predictive models (e.g., neural networks, other machine learning models, or other predictive models) to generate graph data models or other data models, generate graphs or other data structures, etc. , may facilitate predicting information, determining query plans, or other operations about such data models or structures. The training data used to train a predictive model includes (i) the inputs provided to the predictive model (e.g., inputs provided to and processed by other predictive models or other inputs); (ii) its reference outputs derived from predictive model processing of such inputs (e.g., user-confirmed or user-provided outputs; outputs confirmed by processing such inputs by one or more predictive models; corresponding predictions; (iii) a reference representation of an output not derived from the processing of a predictive model of such input; (e.g., a user indication or other reference indication that such output is inaccurate); or (iv) other training data.

In some embodiments, the model management subsystem 114 includes modeling information associated with a graph data model set (including one or more graph data models), a graph A collection of data representations or other information for each graph data model may be obtained for conversion to a data representation. Model management subsystem 114 may provide modeling information, collections of data representations, or other information to the predictive model to train the predictive model.

As an example, modeling information may include (i) converting a data representation (e.g., a non-graph data representation or other data representation incompatible with a particular graph database) into a graph data representation that is compatible with a graph database; (2) a template (associated with a template) for matching an incompatible data representation to at least one of the available templates that can be used to convert an incompatible data representation to a compatible graph data representation; (3) other modeling information. For each graph data model in the graph data model set and the incompatible data representations (that the graph data model transforms), the model management subsystem 114 identifies one or more of the graph data model's templates and a compatible data representation. A data representation free of data may be provided to a predictive model to predict one or more additional templates of the graph data model's templates. For example, in one use case, the predictive model predicts additional templates as part of training the predictive model, even if such additional templates already exist as part of the graph data model. In this case, in one use case, the predictive model can be configured to use additional Templates may be predicted.

As a further example, for each graph data model in the set of graph data models, the model management subsystem 114 adds additional templates of the graph data model's template to the predictive model as reference feedback to the predictions of the additional templates made by the predictive model. may be provided to train the predictive model. The predictive model uses addition templates as "reference templates" to make additions that the predictive model makes (e.g., templates generated by the predictive model based on other templates provided as inputs to the predictive model). The predictions of the template may be evaluated. Based on the evaluation of its own predictions, the predictive model (by adjusting the weights of the predictive model's parameters or other parts of the predictive model depending on whether or how accurate the prediction was) One or more portions of the predictive model may be updated. In one use case where the predictive model is a neural network, the neural network may update one or more layers of the neural network based on the neural network's evaluation of the additional template's predictions. As an example, a neural network uses forward or backward propagation techniques to create one or more layers (e.g. hidden layers between the input and output layers of the neural network or other layers of the neural network). The weights of the neural units within the template may be reset or changed based on the accuracy of the predictions (e.g., based on whether or how accurate the predictions of one or more additional templates were).

In some embodiments, once the predictive model is trained (or updated based on such training), model management subsystem 114 may use the predictive model to generate a graph data model. . In one example, the model management subsystem 114 includes information in the predictive model that is a graph data model or that can be used to generate a graph data model (e.g., a template, pattern, rule, or other information for generating a graph data model). ) can be generated. As a further example, with respect to converting non-graph data representations to graph data representations, model management subsystem 114 may provide a collection of non-graph data representations from a non-graph database as input to a predictive model. In response to such input, the predictive model may output one or more templates, patterns, rules, or other information of the graph data model. The model management subsystem 114 uses the templates, patterns or rules of the templates, or other information to enable the graph data model to convert the non-graph data representation (from the non-graph database) into a graph data representation compatible with the graph database. A graph data model may be generated such that the graph data model is configured to convert the data into a data model.

In some embodiments, data management subsystem 112 may utilize one or more predictive models to predict information in nodes, edges, or other data representations to generate graphs or other data structures. Often, for example, a data structure may be generated, one or more portions of a data structure may be modified, or a data structure may be supplemented based on predictive information about nodes, edges, or other data representations. In some embodiments, data management subsystem 112 may obtain graph information related to nodes or edges of the graph, other information regarding data points, or information regarding relationships of other data sources. Data management subsystem 112 provides such graph information or other information to a predictive model to train the predictive model and predict information about additional or alternative nodes, edges, or other data representations (e.g., to create new graphs or other data structures, modify existing graphs or other data structures, or supplement existing graphs or other data structures).

In some embodiments, data management subsystem 112 may obtain one or more data representation sets and provide the data representation sets to a predictive model for training the predictive model. As an example, a data representation set may include nodes, edges, or other data representations. Data management subsystem 112 predicts one or more nodes or edges of the data representation set as an input to a predictive model for the predictive model to predict one or more additional nodes or edges of the data representation set. May be provided to the model. As used herein, providing a node or edge also includes providing data represented by the node or edge. In one use case, for example, during training of a predictive model, even if such additional nodes or edges already exist as part of the data representation set (e.g., as part of an existing graph). Alternatively, the predictive model may predict additional nodes or edges. Thus, in one use case, the predictive model may depend on any of the additional nodes or edges without depending on the additional nodes or edges (e.g., without relying on at least some of the additional nodes or edges). additional nodes or edges may be predicted.

As a further example, for each data representation set of the multiple data representation sets, the data management subsystem 112 may provide the additional nodes or edges of the data representation set to the predictive model as reference feedback for the prediction of the additional nodes or edges made by the predictive model in order to train the predictive model. The predictive model may use the additional nodes or edges as "reference nodes or edges" to evaluate a prediction of the additional nodes or edges (e.g., nodes or edges generated by the predictive model based on other nodes or edges provided as inputs to the predictive model). Based on the evaluation of its prediction, the predictive model may update one or more parts of the predictive model (by adjusting the parameters of the predictive model or weights of other parts of the predictive model depending on whether or how accurate the prediction was). In one use case where the predictive model is a neural network, the neural network may update one or more layers of the neural network based on the neural network's evaluation of the prediction of the additional nodes or edges.

In some embodiments, once a predictive model is trained (or updated based on such training), data management subsystem 112 uses the predictive model to create data representations for new or existing data structures. may be created or modified. By way of example, data management subsystem 112 may use predictive models to create new nodes or edges in new or existing graphs, modify one or more nodes or edges in existing graphs, or perform other operations. You may also execute In some embodiments, data management subsystem 112 may perform a traversal of the graph to process nodes or edges of the graph (e.g., a predictive model may traverse the graph and identify nodes during the traversal. or edges as input, one or more agents crawl the graph to extract nodes of the graph during traversal, and provide the extracted nodes or edges as input to a predictive model). . Once we have such an input resulting from a traversal, the predictive model can use the ) may generate new nodes or edges for the graph.

In some embodiments, in response to generating a data representation (e.g., a node, edge, or other data representation) by the predictive model, data management subsystem 112: (i) subsequent to generating the data representation; and (ii) A data representation may be automatically added to a graph or other data structure without user input, without indicating whether the data representation is to be added. As an example, data management subsystem 112 may add the data representation as a new data representation of a data structure, or add the data representation as an alternative data representation of an existing data representation of a data structure, or A data structure may be modified based on the data representation.

On the other hand, in some embodiments, the addition of new data representations (or the decision not to add new data representations) to a graph or other data structure may be made based on user input that is subsequently performed. As an example, a predictive model may generate nodes or edges, and data management subsystem 112 may obtain the nodes or edges from the predictive model and notify a user regarding the nodes or edges. In one use case, the notification is a prompt to review the node or edge, confirm or deny use of the node or edge as a new or replacement node or edge for a graph, or Other notifications may also be included. In response to a user's consent to add a node or edge (e.g., as a new node or edge, as a replacement node or edge, etc.), data management subsystem 112 adds the new node or edge to the graph. may be added to. Alternatively, in response to the user's refusal to add a node or edge, data management subsystem 112 may decide not to add the new node or new edge to the graph.

In some embodiments, data management subsystem 112 may provide information indicating user acceptance or user rejection as reference feedback regarding the nodes or edges generated by the predictive model. A predictive model may use reference feedback to evaluate node or edge predictions. A predictive model can adjust the weights of its parameters or other parts of the predictive model based on its evaluation of the predictions it has made (e.g., depending on whether or how accurate the prediction was). (by updating one or more parts of the predictive model). In one use case where the predictive model is a neural network, the neural network may update one or more layers of the neural network based on the neural network's evaluation of predictions of additional nodes or edges.

Continuing with the information about movie 3 and movie 4 from FIG. 5, FIG. 6 shows two different graph data representations 600 and 602 generated based on templates from trained predictive models in accordance with one or more embodiments. An example is shown. Graph data representations 600 and 602 can be broadly thought of as updates to graph data representation 504 from FIG. 5 that were generated for additional training of the predictive model. Graph data representations 600 and 602 may be generated by data management subsystem 112 (FIG. 1) based on templates generated by predictive models. For example, graph data representation 600 includes nodes 520 and edges 518 related to movie 3 shown in graph data representation 504 from FIG. Graph data representation 600 also includes nodes 604 and edges 606 related to additional box office receipts in movie 3 (eg, Australian box office receipts). Information related to nodes 604 and edges 606 was included in data representation 500 of FIG. 5 but not in graph data representation 504. However, training the predictive model may cause the predictive model to generate additional templates, and these additional templates may be used to generate additional graph data representations, such as graph data representation 600.

A further example of a graph data representation 602 is shown. Graph data representation 602 includes nodes 520 and 604 and edges 518 and 606 related to movie 3. Graph data representation 602 also includes nodes 608 and edges 610 related to movie 4. Again, in this example, information related to nodes 608 and edges 610 was included in data representation 500 of FIG. 5 but not in graph data representation 602. However, training the predictive model may cause the predictive model to generate additional templates, and these additional templates may be used to generate additional graph data representations, such as graph data representation 602.

In one usage example, as shown in FIG. 7, data model 700 is associated with a movie 702. Data model 700 includes document nodes 704, location nodes 706, people nodes 708, and organization nodes 710. Document node 704 may include or be associated with information such as movie reviews, movie scripts, movie summaries, etc. (e.g., via other edges and nodes not shown in data model 700). You can leave it there. Location nodes 706 may include or be associated with information such as filming locations, locations depicted in movies, or other information. The person node 708 may include or be associated with the actors appearing in the movie, the author of the movie's script, the movie's director, or other information. Organization node 710 may include or be associated with the production company responsible for the movie or other information. Graph data model 700 may be used to perform logical reasoning.

In another use case, predictive models may be used to form data relationships through inference, as shown in FIG. In step 802, nodes 806, 808, and 810 for two different actors (actor 1, actor 2) and director (director 1), and nodes 812 and 814 for two different movies (movie 5, movie 6); A corresponding edge 816 may be identified. In step 804, the predictive model may conversely infer 820 that if the movie includes an actor, then that actor is an actor in the movie. Stated another way, if edge 816 connects the movie and the actor, then there must be an inverse edge 820 that connects the actor and the movie. Similar reasoning 820 applies to supervision. Additionally, the predictive model may infer that the actor or director was involved in the production of the film (and may establish a corresponding edge indicating so). Since the predictive model knows who worked on the film, the predictive model may make an inference 824 that two people who worked on the same film should be co-stars (and a corresponding (may establish an edge). Finally, in step 804, since the predictive model established relationships between co-stars, the predictive model infers 826 that the actor and director are connected to each other through one or more co-stars (and that (establish the corresponding edge shown).

In another use case, as shown in Figure 9, the predictive model (in the example used herein) determines which movies are relative to each other by inference based on the data model and/or supervised learning by the predictive model. It may also be possible to predict whether they are similar. For example, the first schematic diagram 900 in FIG. 9 shows Movie 5 and Movie 7, both of which are similar to Movie 6. In some embodiments, supervised learning may be used to train the predictive model, as shown in second schematic diagram 902 of FIG. As mentioned above, a given node or edge from the predictive model may be added to the graph based on the user's consent to add the given node or edge. Information indicating user acceptance may be provided to the predictive model by the model management subsystem 114 (FIG. 1) as reference feedback regarding the predictive model's generation of a given node or edge. In some embodiments, a user's refusal to add a given node or edge may be obtained. A given node or edge may not be added to the graph in response to the user's refusal. Information indicating that the user has rejected may be provided to the predictive model as reference feedback regarding that the predictive model generated a given node or edge. As shown in schematic diagram 902, nodes 904, 906, and 908 for Actor 1, Director 2, and Keywords are connected to node 910 for Movie 5 by corresponding edges 912, 914, and 916. In some embodiments, the predictive model includes a rating node 918 for movie 5 (connected via edge 920) and a classification node 922 for movie 5 (connected via edge 924). The information may be configured to make suggestions, and these pieces of information may be accepted or denied by the user.

In some embodiments, similarity values or other measures of similarity between nodes may be determined. As an example, as shown in FIG. 10, programming code may be configured to determine similarity values between nodes of a graph. In this example, the nodes correspond to movies 5, 6, and 7 shown in FIG. 9, respectively. As shown in Figure 10, the prediction is that movie 6 has a similarity value of 0.323291 to movie 5, movie 7 has a similarity value of 0.290015 to movie 5, and movie 8 has a similarity value of 0.290015 to movie 5. may be used to predict that movie 5 has a similarity value of 0.159687.

In some embodiments, model management subsystem 114 may cause the predictive model to construct new nodes or edges, learn rules, or learn ontology matches. In one or more embodiments, examples of generating new nodes or new edges, learning rules, and learning ontology matches in accordance with a movie theme are illustrated in FIG. 11 as examples 1100, 1102, and 1104, respectively. ing. As shown in example 1100, the predictive model generates nodes such as a movie and its corresponding title, the year the movie was made, the movie's director, the movie's producer, and one or more actors who appeared in the movie. Good too. In some embodiments, the predictive model may generate these nodes based on text or other information provided to the predictive model. (In some embodiments, as shown in Figure 3,
The text may have undergone metadata extraction, text extraction, graph extraction, or other extraction processes performed by model management subsystem 1104 or other subsystems of server 102). As shown in example 1102, the predictive model may learn rules. Predictive models may learn rules based on nodes and edges of the graph, paths through nodes, or other information. In this example, the predictive model may learn that a movie is a crime genre movie if it is associated with violence. Finally, as shown in example 1104, the predictive model may learn ontology matches. An ontology match may include different words that refer to the same or similar concepts. As shown in this example, the predictive model may learn that "actor" and "movie star" correspond to the same concept. Ontology Match uses three types of similarity metrics to find mappings between terms: syntactic similarity, semantic similarity, and structural similarity. Syntactic similarity uses techniques such as edit distance, fuzzy string matching, or trigram cosine similarity to assess the similarity of two terms based on the characters in their labels. For example, the similarity between the words "birthday" and "date of birth" is detected. Semantic similarity takes into account the meaning of a label by utilizing a manually curated lexical database (eg, WordNet) or a separately trained word embedding model. Word embeddings map words to vector representations based on word usage, such that words used in similar ways in the training dataset have similar representations. This makes it possible to detect similarities between syntactically different labels such as "actor" and "movie star." Structural similarity then looks at how terms are defined within the schema. For example, one schema defines the relationship "starredIn" between the concepts "Actor" and "Film," while another schema defines the relationship "starredIn" between the concepts "Actor" and "Movie." ” may be defined as a relationship “workedOn”. Once the mappings between concepts (the mappings of "actor-movie star" and "movie-movie") are established, the similarity between "starring" and "starring" is based on relationships having the same source and target types. Detected.

In some embodiments, request subsystem 116 may utilize one or more predictive models to determine one or more query plans. In some embodiments, request subsystem 116 includes path information (e.g., a path returned by a path query as described herein), query plan information (e.g., Information indicating past query plans, the actual cost of executing past query plans, etc.) or other information may be obtained from one or more historical databases or other sources. The request subsystem 116 provides path information, query plan information, or other information to the predictive model to train the predictive model and respond to one or more requests (e.g., requests from users, predicted requests, or other requests). information about one or more query plans that will be used to respond to a request (such as an automatically generated request). In some embodiments, once the predictive model is trained (or updated based on such training), the request subsystem 116 uses the predictive model (e.g., to respond to a request from a user). one or more query plans may be generated (e.g., in real time, in response to predictions of requests, etc.).

[Query prediction, storage and response]

In some embodiments, request subsystem 116 may be configured to predict that data requests will occur in the future. As an example, a request may include a query submission (or client-initiated query), an update request, or other request related to a client-initiated query. In some embodiments, request subsystem 116 may predict requests for query results and obtain a subset of results in response to the request prediction. Data management subsystem 112 may store the subset of results in temporary data storage (eg, a server cache, web cache, memory cache, or other temporary data storage). In some embodiments, data management subsystem 112 may convert a subset of results into one or more subgraphs (e.g., if the results are not in an appropriate graph format) and store the subgraphs in temporary data storage. good. When a predicted request (or a future request that matches a predicted request) occurs, the request subsystem 116 retrieves one or more subgraphs from temporary data storage and retrieves them for the predicted request that occurred. You may respond using subgraphs. This way, for example, by temporarily storing results in a transformed format (before a particular request occurs), you can significantly reduce latency and other delays in fully responding to a request. It may be possible to reduce it.

In some embodiments, request subsystem 116 predicts that requests will occur in the future based on past queries (e.g., past queries or other past queries that are compatible with a graph data model). It's okay. For example, request prediction may include request history information, which is information indicating one or more past queries, information indicating the frequency of each request (for example, frequency of each past query, update request related to the past query, etc.), It may also be based on information about users or client devices that initiated past requests, or other information. In some scenarios, at least a portion of the requested query results may be obtained based on a prediction of the request prior to obtaining the request from the client device in the future. The obtained query results may be stored (e.g., in a temporary data storage such as a server cache, web cache, memory cache, or other temporary data storage) in anticipation of future requests, and the stored query The results may be used to respond to future requests as they occur.

In some embodiments, in response to predicting a data request, request subsystem 116 may generate one or more graph queries based on one or more parameters of the predicted data request. By way of example, parameters may include keywords, content items or their identifiers/locations (e.g., content IDs, hyperlinks or other pointers to content items, etc.), logical operators (e.g., logical AND operators, logical OR operators, may include one or more search parameters such as a logical NOT operator or other logical operators) or other parameters. In one use case, if the content item is an image, the image is searched for similar images, search for content items that have the same image or similar images, search for content items that have a similar concept to the concept in the image, and search for content items that have a similar concept to the concept in the image. or can be used to search for other results. In another use case, if the content item is a video, the video can be searched for similar videos, search for content items that have the same video or similar videos, or search for content items that have similar concepts to the concepts in the video. Can be used for searching or finding other results.

In some embodiments, the request subsystem 116 executes at least one of the graph queries (e.g., compatible with one or more SQL or other non-graph databases) based on a graph data model. It may also be converted to the above non-graph query. As an example, a non-graph query may be executed to obtain a data subset from one or more non-graph databases. In some embodiments, at least one of the graph queries may be executed to obtain a data subset from one or more graph databases. As an example, a portion of a graph query may be converted to a non-graph query to obtain results (related to a predictive data request) that are stored in a non-graph database. However, if other graph queries are related to results stored in the graph database, there may be no need to convert the graph query to a non-graph query. In some scenarios, for example, request subsystem 116 determines where results associated with a graph query are stored (e.g., whether the results are stored in either a non-graph database or a graph database) and Based on this determination, a non-graph database or a graph database from which relevant results can be obtained may be selected. Based on the selection of the non-graph database (to obtain at least some results related to the given graph query), the request subsystem 116 converts the graph query into a non-graph query that is compatible with the non-graph database. You may.

In some embodiments, the data management subsystem 112 may store a graph, one or more subgraphs (e.g., collectively comprising the graph or a portion thereof), or one or more graph data representations (e.g., , nodes, edges, etc. that collectively constitute a graph, subgraph, or part of a graph/subgraph). As shown herein, in some embodiments, the data management subsystem 112 stores data representations (e.g., stored in SQL tables or other data sources) (e.g., with a particular graph database). One or more graph data models may be used to convert the data into a compatible (compatible) graph representation. In response to predicting that a data request will occur in the future, request subsystem 116 may obtain one or more data subsets from one or more data sources that are predicted to be requested by the future data request. good. Upon obtaining such data subset, data management subsystem 112 may process the data (e.g., in a non-graphical format or other incompatible representation) in accordance with one or more techniques described herein. The graph data model is used to transform the subset into one or more graph data representations (e.g., compatible with a particular graph database) or one or more subgraphs containing the graph data representations. It's okay.

In some embodiments, one or more data subsets are obtained from one or more non-graph data sources (e.g., a SQL database or other non-graph data sources) in response to predicting the data request. One or more data subsets may be obtained from one or more graph data sources (eg, graph databases or other graph data sources). As an example, a data subset (from a non-graph data source) may be obtained as a non-graph data representation (eg, an SQL row or column or other non-graph data representation) from a non-graph data source. Additionally, other graph data representations (from the graph data source) such as nodes, edges, or other graph data representations (e.g., compatible with a particular graph database, incompatible with a graph database, etc.) Data subsets may be obtained. Data management subsystem 112 manages one or more subgraphs representing multiple data subsets (from non-graph data sources) and one or more subgraphs representing multiple other data subsets (from graph data sources). may be generated. As an example, for data subsets obtained from non-graph sources, data management subsystem 112 uses a graph data model to convert the non-graph data representation to a graph data representation that is compatible with the graph database and The representation may be compiled into the above subgraphs. As another example, for a data subset obtained from a graph source, data management subsystem 112 uses a graph data model to compile the graph data representation (of such data subset) into subgraphs of other representations. You may. If the graph data representation is in a format that is not compatible with the graph database, data management subsystem 112 uses the graph data model to convert the incompatible data representation to a compatible graph data representation. However, compatible graph data representations may be compiled into subgraphs of other representations.

In some embodiments, data management subsystem 112 may store subgraphs (derived from non-graph data subsets) and other subgraphs (derived from graph data subsets) in temporary data storage. In response to retrieving the next data request that matches the predicted data request, request subsystem 112 retrieves the subgraph, other subgraphs, or other information from temporary data storage and The information may be used to respond to the next data request. In some embodiments, the request subsystem may extract a data subset from the nodes, edges, or other graph data representations of the obtained subgraph and return the extracted data subset in response to a subsequent data request.

In some embodiments, in response to obtaining a next data request that matches the predicted data request, request subsystem 116 generates a query plan for responding to the next data request. Good too. As an example, a candidate query plan may be selected from a collection of query plans (or template query plans) and then modified to generate a query plan specific to the next data request. In some embodiments, if the subgraph or other information related to the predicted data request is stored in the temporary data storage, (i) retrieving the subgraph or other information from the temporary data storage; (ii) ) obtaining other information from other data sources (eg, graph databases, non-graph databases, etc.). As an example, based on the query plan, request subsystem 116 may extract one or more subgraphs related to the predicted data request from temporary data storage to one or more other data subsets (e.g., related to the predicted data request). related to subsequent data requests, etc.) may be obtained from one or more data sources. Request subsystem 116 may use the subgraph (or the data set that the subgraph represents) and other data subsets to respond to subsequent data requests.

In some embodiments, a query (e.g., a graph query, a non-graph query) executed in response to a prediction of a data request is based on the predicted data that would have been retrieved from the client device if the predicted data request had been obtained from the client device. It may be part of the set of queries that would have been executed in response to the request. In some embodiments, the prediction of the data request may not result in execution of one or more other queries in the set of queries. As an example, querying for others may not result from prediction of the request.

In some embodiments, the subset of results for a query may be a portion of a set of results that would have been obtained to respond to the request if the request had been obtained from the client device. For example, if the set of results is all results that would have been provided on a first web page (e.g., a list or other presentation of the most relevant results) in response to the client device, the subset of results may be a portion of those results that would have been provided on the first web page. In another use case, the set of results may be all results that would have been obtained to respond to the request if the request had been obtained from the client device. As an example, a determination may be made (e.g., based on frequency information, cost information, etc., as described herein) not to store the other subset (of the set of results) in the temporary data storage, even if the other subset (of the set of results) has been obtained (e.g., via one or more queries in response to a request prediction).

In some embodiments, the acquisition or storage of subgraphs or other data (or the decision not to acquire or store other subgraphs or other data) may be based on frequency information, cost information, or other information. In some embodiments, request subsystem 116 may perform selection of a subset of queries to be executed (over other queries) based on frequency information, cost information, or other information. The frequency information may include information indicating the frequency of requests matching the request or other information. The cost information may include information indicating the cost of storing data in temporary data storage, information indicating the cost of executing each query, or other information. Such costs include, for example, monetary costs, computer resource costs (eg, network resource usage such as bandwidth, and other computer resource costs), and other costs.

In response to the request prediction, a cost/benefit analysis may be performed, for example, to determine which results will be obtained or saved, and how much of the results will be obtained or saved. In one embodiment, the request subsystem 116 determines whether (or which queries) to perform, the respective costs (e.g., the cost of querying the data source for data), and the costs derived from those queries. Benefits of the results (e.g., how often requests match predicted requests, which results should be prioritized over others based on the preferences of the person making the request, etc.), and how those results can be stored in temporary data storage The determination may be made based on the respective cost of storing the data, or other criteria. A further use case could be to assign ( (before execution) may be assigned a score. As an example, the lower the cost of querying a data source for data, the higher the score (compared to the scores of other queries) assigned to the corresponding query. The more frequently requests match the predicted request, the higher the score assigned to the query related to the predicted request. The results obtained from one query are more likely to be presented (on the user interface) to the person making the request (e.g., than other results obtained from other queries based on the preferences of the person making the request). ), the higher the query's score (compared to the scores of other queries). Based on the scores assigned to each, request subsystem 116 may determine whether to execute one or more of the queries. As an example, request subsystem 116 may select a subset of queries to execute based on one subset of such queries having a greater score than another subset.

In another use case, even if results are obtained, the temporary request subsystem 116 determines the results based on the cost of storing each of these results in temporary data storage, the merits of each of these results, or other criteria. (or how much of the result should be stored). A further use case could be to assign a score to each result (e.g., a subset of results) based on the cost of storing each of these results in temporary data storage, the merits of each of these results, or other criteria. good. As an example, the lower the cost to store a particular subset of results may have a greater impact on assigning a higher score to the subset of results. The more frequently requests match the predicted request, the higher the score assigned to the results associated with the predicted request. The more likely a particular subset of results is to be presented (on the user interface) to the person making the request (e.g., over other results based on the preferences of the person making the request), the more likely such results are may influence the high scores of a subset of Based on the scores assigned to each, request subsystem 116 may determine which of the results are stored in temporary data storage. Request subsystem 116 may select a subset of results to be stored, e.g., based on a subset of results having a greater score than other subsets of results (e.g., obtained from an executed query). .

In some embodiments, results are obtained or stored in response to predictions of one or more requests (as described herein), but not in response to predictions of other requests. (e.g., even if the probability of each of those other requests to occur meets a certainty threshold), no results may be obtained or stored. As an example, the request subsystem 116 may respond to predictions of requests based on cost/benefit analysis performed on the predicted requests (e.g., based on frequency information, cost information, or other information). You may decide not to run any queries either. As another example, the request subsystem 116 determines the results obtained from the request prediction based on a cost/benefit analysis performed on the predicted requests (e.g., based on frequency information, cost information, or other information). You may decide not to store it.

In some embodiments, model management subsystem 114 may be configured to obtain request history information and provide request history information to the predictive model for training the predictive model. Request history information may include (i) a collection of past requests (e.g., requests submitted by a user for data); (ii) a collection of past queries generated from past requests (e.g., a collection of past queries that are compatible with a graph data model). a graph query configured to have a graph query), (iii) timing information indicating the time at which the past request or past query was obtained, (iv) frequency information indicating the frequency of the past request or past query, (v) sending the past request. User information (such as a non-personally identifiable user identifier or other identifier) or type of user (such as age, gender, location, or other classification) that indicates which user has submitted a request, and which past requests the user or type of user has submitted. or (vi) other information may be included.

In some embodiments, the predictive model may be configured to obtain at least one type of request history information and predict at least another type of request history information based on the obtained information. As an example, model management subsystem 114 may provide timing information (indicating the time at which the past request or past query was obtained), frequency information (past request timing information, frequency information (indicating the frequency of past queries), user information (indicating the user or type of user who submitted the past request), or other information related to the past request or past query. , user information or other information about past requests or queries may be provided as reference feedback to the prediction, by which a predictive model may be trained. A predictive model may use reference feedback to evaluate the information it has predicted. As another example, for timing information, frequency information, or user information provided as input to a predictive model, the model management subsystem 114 may compare past requests or past queries associated with the input information to requests or queries made by the predictive model. A predictive model may be trained by providing reference feedback to the prediction of the query. A predictive model may use reference feedback to evaluate predictions of requests or queries that it has made. A predictive model can adjust the weights of its parameters or other parts of the predictive model based on its evaluation of the predictions it has made (e.g., depending on whether or how accurate the prediction was). (by updating one or more parts of the predictive model). In one use case where the predictive model is a neural network, the neural network may update one or more layers of the neural network based on the neural network's evaluation of the additional template's predictions.

In some embodiments, once a predictive model is trained (or updated based on such training), model management subsystem 114 uses the predictive model to (i) (ii) timing information for these requests or queries, (iii) frequency information for these requests or queries, (iv) user information for these requests or queries, or (v) other information for these requests or queries. Information may also be predicted. By way of example, such a prediction may include one or more parameters of the predicted request, such as search parameters (e.g., keywords, content items or identifiers/locations thereof, logical operators, etc.) or other parameters. . As another example, such predictions may include one or more of the times of the predicted request (or the next request that matches the predicted request), the frequency of the predicted request, and the expected frequency of sending that request. may include user or user type or other predictions. Based on such predictions, request subsystem 116 retrieves one or more subsets of the results and stores the results in temporary data storage (e.g., server cache, web cache, memory (e.g., in the form of a transformed subgraph compatible with a graph database or other format). When a predicted request (or a future request that matches the predicted request) occurs, the request subsystem 116 retrieves one or more results from temporary data storage and uses the retrieved results to May respond to predicted requests.

[Query set optimization]

In some embodiments, optimization subsystem 118 may be configured to reduce query-related resource usage in the data retrieval process.
In some embodiments, the optimization subsystem 118 includes a query set derived from a data request (e.g., an explicit request from a user or other request), such as a graph query (or other request related to the data request). The resource usage associated with such queries may be reduced by optimizing the query set in which the queries) are transformed. In some embodiments, such query set optimizations include removing query operators that link multiple queries from the query set, merging multiple queries in the query set into a single query, or may include removing multiple queries from the query set or other optimizations. Such optimizations are based on predictions of satisfiability problems (e.g., related to the combination of multiple results obtained from a particular query), incompatibility problems, or other problems. may be performed to avoid or lessen the problem or the adverse effects of such problem.

In some embodiments, request subsystem 116 obtains and processes multiple data requests (e.g., from one or more user devices) and determines whether the multiple data requests seek common subject data ( For example, it may be determined whether at least some of the data requested by the plurality of data requests is the same among the plurality of data requests. Based on determining that the data requests seek common data of interest, request subsystem 116 may issue one or more queries (e.g., as part of a query set) to respond to all data requests. ) may be generated. wherein each of the queries is configured to obtain at least a portion of the data commonly sought by multiple data requests, and where one such query is used to obtain the common data portion. can. In some embodiments, the request subsystem 116 is configured such that at least one of the plurality of queries obtains a first set of commonly sought data from the first source; The plurality of queries may be generated such that at least another one of them is configured to obtain a second set of commonly sought data from a second source. As an example, a predetermined query for retrieving a first set of commonly sought data from a first source is compatible with the first source; may be configured to be incompatible with the second source. As another example, a predetermined query for retrieving a second set of commonly sought data from a second source may be compatible with the second source; , but may be configured to be incompatible with the first source. In response to obtaining a set of commonly desired data from different sources (e.g., a first source, a second source, etc.), request subsystem 116 retrieves a set of commonly desired data. may be configured to respond to each data request by combining the data requests and returning the combined set.

In some embodiments, the request subsystem 116 determines whether each of the plurality of data requests (e.g., obtained from one or more user devices) is associated with two or more attributes common to all data requests. It may be determined to obtain two or more values. As an example, the data request may be determined to (i) request the names of multiple individuals within group A, the names of multiple individuals within group B, etc., or (ii) request the names of multiple individuals within group A, etc. The addresses of multiple individuals in Group B, the addresses of multiple individuals in Group B, etc. may be determined at once. In one use case, the first data request may seek the name and address of a first individual of one of the plurality of groups, and the second data request may seek the name and address of a first individual of one of the plurality of groups. An individual's name and address may be requested. In some embodiments, (i) the value associated with the first common attribute (e.g., name) is retrievable from the first data source; and (ii) the value associated with the first common attribute (e.g., address) is retrievable from the first data source. Request subsystem 116 may generate one or more queries (eg, as part of a query set) based on the determination that the value associated with . wherein at least one of the plurality of queries is configured to retrieve a value associated with the first common attribute from the first data source, and at least another one of the plurality of queries is , configured to obtain a value associated with the second common attribute from the second data source. In some embodiments, for each data request, request subsystem 116 combines the requested value obtained from the first data source and the requested value obtained from the second data source; May respond to data requests by returning a combined request value. In a given scenario, there may be many data requests (e.g., tens of requests, hundreds of requests, thousands of requests, etc.), each asking for the name and address of a different person, but Even if one query is generated to retrieve names from a first data source for all data requests and another query is generated to retrieve addresses from a second data source for all those data requests. good. After running the two queries to retrieve at least a portion of the name and address, each person's name and address are combined into a corresponding data request (i.e., a request for that particular person's name and address). A reply may be sent in response.

In some embodiments, after receiving a first request for one or more values associated with one or more attributes, the request subsystem 116 may wait a predetermined time to elapse before generating or executing one or more queries configured to retrieve data to respond to the first request. This waiting period allows other requests (requesting values associated with at least one attribute common to the attribute sought in the first request) to be retrieved, thereby reducing the overall number of queries that need to be executed to respond to the first request and the other requests. In some embodiments, the request subsystem 116 may determine whether a predetermined time has elapsed (e.g., the time since receiving the given request). If the predetermined time has not elapsed, the request subsystem 116 may delay generating or executing one or more queries configured to retrieve data for the given request. On the other hand, if the predetermined time has elapsed, the request subsystem 116 may determine which other requests have been retrieved after receiving the given request. For the given request and the other requests that request values associated with at least one attribute as a value sought in the given request, the request subsystem 116 may generate and execute one or more queries configured to retrieve values associated with the common attribute sought by these requests.

In some embodiments, the request subsystem 116 indicates (i) a collection of past requests, (ii) a collection of past queries generated from past requests, and (iii) a time at which the past requests or past queries were obtained. (iv) frequency information indicating the frequency of past requests or queries; (v) user information indicating the user or type of user who sent the past request and which past requests were sent by the user or type of user; The predetermined time (as the waiting time described in this specification) may be set based on request history information such as user information indicating , or (vi) other information. In some embodiments, a predetermined amount of time may be set for each category of request, and the request may be made based on the content sought by the request, the user or the type of user (e.g., age, gender, location, or other information of the user). classification) or other criteria. In some embodiments, such a predetermined time may be determined for a given request in response to obtaining the request (eg, in real time). As an example, in response to retrieving a request, request subsystem 116 determines the category of the request, which is then used as a wait time (e.g., to enable retrieval of more similar requests). A predetermined amount of time may be determined.

As indicated herein, in some embodiments, the request subsystem 116 performs a single process, such as predicting requests that each seek a value associated with a common (e.g., common to all requests) attribute. It may be possible to predict that more than one request will occur in the future. Such predictions may be based on request history information, as described herein. In some embodiments, request subsystem 116 determines whether each category of requests is based on predictions associated with future requests (e.g., predictions regarding when or how often requests in each category will occur). A given time may be set. As also described herein, a predictive model may be trained based on request history information to output information indicative of such predictions. In some embodiments, the predictive model may be configured to evaluate its own predictions, and based on such evaluation, the predictive model may determine whether or not the prediction was accurate (e.g., whether or how accurate the prediction was). One or more parts of the predictive model may be updated depending on whether it was accurate (by adjusting the parameters of the predictive model or the weights of other parts of the predictive model).

In some embodiments, request subsystem 116 may obtain a data request from a user device or other source and generate one or more queries based on the data request. In some embodiments, request subsystem 116 may generate a graph query based on the data request. In response to obtaining the graph query, optimization subsystem 118 converts the graph query into a query having multiple queries and query operators (e.g., union, join, or other query operators) that link the queries. May be converted to a set. In some embodiments, the graph query is compatible with a graph database, and the graph query may be used to generate multiple queries over a query set that is compatible with one or more target data sources. . As an example, a query compatible with the data source may be executed to retrieve data from the data source in accordance with the data source's database management system. In some embodiments, the plurality of queries may be compatible with the target data source, but not compatible with the graph database (whereas graph queries are compatible). In some embodiments, based on the data request, request subsystem 116 may generate a first recursive query (eg, a graph recursive query compatible with a graph database). Request subsystem 116 may convert the first recursive query into one or more second recursive queries (eg, for a query set). As an example, each of the second recursive queries may be configured to be compatible with the target data source, such that the second recursive query is executed by the target computer system (e.g., hosting the target data source). When one of the recursive queries is executed, the executed recursive query causes the target computer system to generate multiple queries from the second recursive query, and executes the multiple queries to obtain data related to the data request from the target data source. Execute.

In some embodiments, with respect to the query set, optimization subsystem 118 predicts one or more problems associated with one or more portions of the query set and optimizes the query set based on the predicted problems. The query set may be updated by performing one or more optimizations on the query set. If such an updated query set is executed to satisfy the graph query (and thus the data request), the predicted problem or the negative impact of the predicted problem may be avoided or mitigated.

As an example, a data request may ask about an employee's salary and the projects assigned to that employee. A data request can be made (for example, for a graph where the "employee" and "salary" nodes are connected by a "hasSalary" edge, and the "employee" and "project" nodes are connected by a "worksOn" edge). The data representation may be written as a graph query with the following two patterns (reflecting how the results of the query are stored):
? Employee: hasSalary? Salary? Employee: worksOn? project

In some use cases, with respect to the example above, a mapping (or a template defined by such a mapping) may point to a source table in an RDBMS that contains payroll information or project information, and the optimization subsystem 118 Mappings (along with database-specific query translators) may be used to convert graph queries into query sets in the language of the database. In some cases, there may be multiple sources for each of the relationships, resulting in a query set with multiple query operators. For example, an employee may be working on multiple projects, and these projects may be stored in multiple tables. In some cases, by converting a graph query into a queryset (to get results from an RDBMS), a UNION for the sources corresponding to one pattern is generated, and the orthogonal product of the UNION for each pattern is generated as a SQLjoin. Good too. If there are N patterns in the graph query and M sources for each term, the query transformation may generate N×M UNIONs. Optimization subsystem 118 may reduce the number of UNIONs, simplify joins within a given UNION, or otherwise optimize the query set.

In some embodiments, optimization subsystem 118 may predict one or more satisfiability problems associated with a query set (e.g., an initial query set transformed from a graph query or other query). good. In some embodiments, based on the prediction of satisfiability problems (related to the combination of results obtained from the two queries), optimization system 118 selects a query from the query set that links the two queries. The query set may be updated by removing operators or performing other optimizations on the query set. As described above, when system 100 executes an updated query set to satisfy a graph query (and thus a data request), system 100: (i) executes one or more portions of the query set; such as the waste of resources used in attempting to combine incompatible results derived from the execution of such querysets, (ii) delays resulting from such executions and attempts, or (iii) any other adverse effects. , satisfiability problems or the negative effects of satisfiability problems can be avoided or reduced.

In some embodiments, if multiple queries in the query set are linked by one or more query operators (e.g., union, join, or other query operators), optimization subsystem 118: One or more sources for obtaining results for the linked query may be determined. If there are at least two sources from which to obtain results for two of the plurality of queries (and such results are initially not compatible with the graph database), the optimization subsystem 118 A template configured to convert a data representation from such a source into a graph data representation compatible with a graph database may be evaluated. If optimization subsystem 118 determines that there are incompatibilities associated with at least two such templates, optimization subsystem 118 determines whether the optimization subsystem 118 is satisfied (associated with combining results from two queries). Other optimizations may be performed on the query set to anticipate degree problems and remove query operators that link two queries from the query set or update the query set. In some use cases, a mapping may define a template for converting rows from an RDBMS into nodes in a graph by creating globally unique identifiers (sometimes referred to as IRIs). good. As an example, an employee with ID 123 may be mapped to an identifier such as "http://example.org/employee/123." If the mappings for the two sources use incompatible templates, optimization subsystem 118 may eliminate the join result, expecting it to be empty. As an example, one template has the form "http://example.org/employee/{ID}" and another template has the form "http://example.org/department/{ID}". , we can conclude that the two templates are incompatible, regardless of the value of the ID. That is, a template has a fixed part and a variable part, such as "http://example.org/employee/" and "{ID}". Although the values of the variables before the query is executed are unknown, if the fixed portions of the templates are inconsistent, the possibility of a combination between the two templates can be excluded using that inconsistency.

In some embodiments, optimization subsystem 118 determines a data type corresponding to a result for a query linked by one or more query operators in the query set, and determines one or more data types associated with the data type. Query set optimization may be performed based on incompatibilities. In some embodiments, the optimization subsystem 118 is used to store the first set of results (from one of the multiple linked queries in the query set) in a graph database. a first data type; a second data type used to store a second set of results (from another one of the plurality of linked queries) within the graph database; It may be determined that they are incompatible. Based on the determination of data type incompatibilities, optimization subsystem 118 predicts sufficiency problems (related to combining results from two queries) and determines whether the two queries Other optimizations may be performed on the queryset to remove query operators that link the queries or update the queryset. In some scenarios, for example, columns in a table may be mapped to primitive values (eg, integers, strings, dates, etc.) in a graph instead of IRIs. If a query attempts to join two incompatible types (e.g., integer and date) from two tables, such an attempt (in this scenario) will fail. Thus, if it is determined that such data types are used to store each of two data sets, optimization subsystem 118 may address satisfiability issues (e.g., two incompatible data types). type join failures) and perform appropriate optimizations (e.g., removing the corresponding query operator, substituting another operator for the query operator, etc.).

In some embodiments, based on the prediction of one or more satisfiability problems associated with the query set, optimization subsystem 118 determines the predicted satisfiability problems and one or more satisfiability problems associated with the query set. Further satisfiability problems above may be anticipated. Optimization subsystem 118 may remove one or more query operators related to additional satisfiability problems or perform other optimizations on the query set. In some embodiments, the first query, the second query, and one or more other queries may be linked by a query operator within a query set. Based on predictions of a first satisfiability problem related to the first query and the second query (e.g., related to combining results from the first query and the second query), The optimization subsystem 118 deletes the first query operator that links the first query and the second query in the query set, or modifies the portion of the query set that includes the first query and the second query. (e.g., excluding the first query operator or making other changes). Based on the prediction of the first satisfiability problem, optimization subsystem 118 determines whether one or more other satisfiability problems (e.g., (relating to a combination of results, a combination of results from the second query and/or other queries, etc.) may be predicted. Based on predictions of other satisfiability problems, optimization subsystem 118 removes second query operators that link two or more of the first query, the second query, or the other queries. , or otherwise modify the portion of the query set that includes the two or more queries (eg, by excluding a second query operator or making other changes). In one use case, optimization subsystem 118 may modify one or more portions of the query set (e.g., modify one or more query operators) to optimize for unsatisfiability propagation. or make other changes). As an example, for a graph query that has two patterns: "?Employee:hasSalary?Salary" and "?Employee:worksOn?Project", the optimization subsystem 118 calculates the satisfiability associated with the two patterns. If we determine that there is a problem (e.g., results derived from queries generated from such patterns cannot be combined), we may predict other problems that may propagate from the pattern. In a further example, in response to determining that two patterns are unsatisfiable (e.g., they cannot be combined), optimization subsystem 118 determines the possibility that other satisfiable patterns exist. Even if there is, it may be determined that unsatisfiability is propagated through the query set. For example, if another pattern is added to the graph query (e.g., when trying to retrieve employee names), the optimization subsystem 118 may assume that no results are returned because the first two patterns could not be combined. You can judge.

In some embodiments, optimization subsystem 118 performs one or more calculations on the query set (e.g., before sending the query to one or more RDBMSs or other database management systems where the query is to be executed). Self-join removal or other optimizations may be performed. In some embodiments, one of the UNION components cannot be eliminated, but may be simplified to improve performance. If the two patterns map to the same source table and that table has a unique key, the optimization subsystem 118 performs a join (e.g., SELECTe1.suffer, e1.salary, e2.suffer, e2.salarayFROMemployeesASe1, employeesase2WHEREe1 .suffer=e2.suffer), a single query (eg, SELECTemployee, salary, projectFROMemployees) may be generated. In this case, the query execution database management system's SQL optimizer may perform this type of transformation, but as more patterns are added to the query, the complexity of the resulting SQL query can be handled by the SQL optimizer. It goes beyond the scope. Queries with too many join conditions or expressions in the SQL WHERE clause exponentially increase the SQL optimizer's search space, causing the optimizer to use heuristics to generate suboptimal query plans. .

In some embodiments, the optimization subsystem 118 may provide the graph query, the corresponding query set (derived from the graph query), or other information to the predictive model to cause the predictive model to predict (i) one or more problems associated with each of the corresponding query sets, (iii) an optimization for each of the corresponding query sets, or (iii) other information. By way of example, such problems may include one or more satisfiability problems (e.g., associated with combining results from a particular query), incompatibility problems, or other problems. Such optimization of a query set may include removing query operators from the query set that link multiple queries, merging multiple queries in the query set into a single query, removing one or more queries from the query set, or other optimizations. In some embodiments, for each corresponding query set, the optimization subsystem 118 may provide one or more reference problems or reference optimizations for the corresponding query set to the predictive model as reference feedback for the prediction of that problem or optimization made by the predictive model to train the predictive model. By way of example, the reference problem or optimization may be provided as reference feedback to cause the predictive model to evaluate the predicted problem or optimization against the reference problem or reference optimization. A predictive model may use a reference problem or reference optimization to evaluate its predictions for the problem or optimization. Based on the evaluation of its predictions, the predictive model may update one or more portions of the predictive model (as described herein).

In some embodiments, optimization subsystem 118 may provide graph queries or other information to a predictive model such that the predictive model predicts a query set for each of the plurality of graph queries. In some embodiments, for each of the plurality of graph queries, optimization subsystem 118 provides a reference query set for the graph query to the predictive model as reference feedback to the predictions of the query set made by the predictive model; A predictive model may be trained. As an example, the reference query set may be provided as reference feedback such that the prediction model evaluates the predicted query set against the reference query set. A predictive model may use a reference query set to evaluate the predictions it has made for the query set. The predictive model may update one or more portions of the predictive model (as described herein) based on the evaluation of the prediction.

In some embodiments, once the predictive model is trained (or updated based on such training), optimization subsystem 118 uses the predictive model to perform (i) graph queries (or or (ii) one or more optimizations to the initial query set. As an example, the optimization subsystem 118 may perform a prediction for (i) a problem associated with the initial query set, (ii) an optimization on the initial query set, or (iii) an optimized query set. A graph query or initial query set may be provided as input to the predictive model. In some use cases, a predictive model in response to such input may include an optimized query set, information indicative of such problem or information indicative of such optimization (e.g. or other instructions). In another use case, optimization subsystem 118 may use information indicative of the problem or optimization to transform an initial query set into an optimized query set.

[Display query response results]

Presentation subsystem 120 may be configured to display query results or other information. The presentation subsystem 120 provides a graph data model template, a prediction of the request for the query results, a subset of the results obtained in response to the prediction of the request, a subset of the results stored in a subgraph in temporary data storage, or other information. The query result may be displayed based on the query result. The displayed query results may include one or more fields in one or more views in a graphical user interface or other interface. A graphical user interface may be displayed on one or more client devices 104 or other computer systems. In some embodiments, the display may include graphics, text, or other representations. In some embodiments, the display may include a submap or other view of the graph data model. In some embodiments, displaying may include providing one or more text and/or graphical fields in various views in a graphical user interface or other display.

In some embodiments, presentation subsystem 120 may be configured to communicate with a graphical user interface to facilitate input or selection of information from a user. For example, as described herein, in some embodiments, a predictive model is created based on user consent entered or selected via a graphical user interface for adding a given node or edge. The given node or edge from may be added to the graph. Information indicating user acceptance may be provided to the predictive model by the presentation subsystem 120 as reference feedback regarding the generation of the given node or edge of the predictive model. In some embodiments, the user's refusal to add the given node or edge may be obtained by presentation subsystem 120 via a graphical user interface. A given node or edge may not be added to the graph in response to the user's refusal. Information indicating the user's rejection may be provided to the predictive model by the presentation subsystem 120 as reference feedback regarding the generation of the given node or edge of the predictive model.

In some embodiments, presentation subsystem 120 communicates with a graphical user interface to extend one or more menus, fields, and/or other objects in or adjacent to one or more other fields. and may be configured to facilitate contractions, pop-ups and/or other displays. In some embodiments, presentation subsystem 120 may generate such a display in response to a user pointing, clicking, or hovering over a particular portion of the display with a pointer or other indicator. . In some embodiments, expanded fields, pop-up fields, additional menu items, and/or other objects display additional complementary or additional information to the user corresponding to the query results.

[Flowchart example]

12-14 are exemplary flowcharts illustrating process operations including methods for enabling the various features and functions of the system described in detail above. The processing operations of each method described below are intended to be illustrative and not limiting. In some embodiments, for example, these methods may be implemented by adding one or more operations not described or by omitting one or more of the operations described. may be done. Additionally, the order in which the process operations of each method are illustrated (and described below) is not intended to be limiting.

In some embodiments, each method includes one or more processing devices (e.g., digital processors, analog processors, digital information processing circuits, information processing analog circuits, state machines, and/or electronic may be implemented in other mechanisms (mechanisms other than these that process information). These processing devices may include one or more devices that perform some or all of the processing of each method in response to instructions electronically stored on an electronic storage medium. These processing devices may include one or more devices configured by hardware, firmware, or software specifically designed to perform one or more of the operations of each method.

FIG. 12 depicts a flowchart of a method 1200 of generating a graph via a predictive model in accordance with one or more embodiments. In some embodiments, the predictive model may include a neural network, machine learning model, or other predictive model.

In operation 1202, first modeling information may be obtained. The first modeling information may be associated with a first graph data model. The first modeling information may include a first template for converting a first data representation that is incompatible with the first graph database to a graph data representation that is compatible with the first graph database. In some embodiments, processing 1202 may include obtaining graph information related to nodes and edges of graphs within a given graph database. The graph information may indicate a data representation set. Each data representation set of the plurality of data representation sets may include nodes and edges connecting the nodes. In accordance with one or more embodiments, process 1202 may be performed by a graph generation subsystem that is the same as or similar to data management subsystem 112.

At operation 1204, one or more of the first templates and the first data representation may be provided to a predictive model. The predictive model may predict one or more additional templates for the first template. The prediction model may be configured to perform predictions for the additional template without relying on the additional template. In some embodiments, processing 1204 may include, for each data representation set of the plurality of data representation sets, providing one or more nodes or edges of the data representation set to the predictive model. The predictive model may predict one or more additional nodes or edges of the data representation set. The prediction model may be configured to perform predictions of additional nodes or edges independently of the additional nodes or edges. According to one or more embodiments, process 1204 may be performed by a graph generation subsystem that is the same as or similar to data management subsystem 112.

At operation 1206, additional templates of the first template may be provided to the predictive model. These templates may be provided as reference feedback for additional template predictions made by the predictive model. A predictive model may be trained by this feedback. In some embodiments, the process 1206 includes, for each data representation set of the plurality of data representation sets, the data representation as reference feedback for predictions of additional nodes or additional edges made by the predictive model to train the predictive model. The method may include providing additional nodes or edges of the set to the predictive model. According to one or more embodiments, processing 1206 may be performed by a model subsystem that is the same as or similar to data management subsystem 114.

At operation 1208, a collection of data representations from a given database may be provided to a predictive model. These data representations allow the predictive model to generate one or more templates for converting a given graph data model into a given graph database's data representation. You can also do this. In some embodiments, the graph data representation of a given graph database includes graph nodes or edges connecting graph nodes. In accordance with one or more embodiments, processing 1208
may be performed by a graph generation subsystem that is the same as or similar to data management subsystem 112.

In some embodiments, processing 1208 may include performing a traversal of the graph through a predictive model to process nodes or edges of the graph. The predictive model may thereby generate new nodes or edges in the graph. New nodes or edges from the predictive model may be added to the graph. A given node or edge from the predictive model may be added to the graph based on a traversal of the graph. In some embodiments, a given node or edge from a predictive model may be added to a graph based on a user's consent to add the given node or edge. Information indicating user acceptance may be provided to the predictive model as reference feedback regarding the predictive model's generation of a given node or edge. In some embodiments, a user's refusal to add a given node or edge may be obtained. A given node or edge may not be added to the graph in response to the user's refusal. Information indicating that the user has rejected may be provided to the predictive model as reference feedback regarding that the predictive model generated a given node or edge.

In some embodiments, method 1200 may further include obtaining second modeling information associated with the second graph data model. The second modeling information may include a second template for converting a second data representation that is incompatible with the second graph database to a graph data representation that is compatible with the second graph database. . In some embodiments, the first data representation and the second data representation may be compatible with the same database. In some embodiments, the first data representation may be compatible with the first non-graph database. The second data representation may be compatible with the second non-graph database and incompatible with the first non-graph database. In some embodiments, the first data representation or the second data representation may be compatible with at least one graph database.

In some embodiments, method 1200 further includes providing one or more of the second templates and the second data representation to the predictive model. The predictive model may predict one or more additional templates for the first template. The prediction model may be configured to predict the additional template of the second template without depending on the additional template of the second template. In some embodiments, the method 1200 provides the additional template of the second template to the predictive model as reference feedback for the prediction of the additional template of the second template by the predictive model to train the predictive model. It further includes:

FIG. 13 is a flowchart illustrating a method 1300 of reducing data retrieval delay through prediction-based generation of data subgraphs in accordance with one or more embodiments.

In operation 1302, a prediction may be made that a data request will occur in the future. Predictions may be based on past queries that are compatible with graph data models. Process 1302 may be performed by a request subsystem that is the same as or similar to request subsystem 116, according to one or more embodiments.

In operation 1304, one or more subgraphs may be generated. Generation of subgraphs may be based on a graph data model. A subgraph may be representative of a data subset of a data set. The data set may be the data that the data request is expected to require. Subgraphs may be generated in response to predictions of data requests. Process 1304 may be performed by a request subsystem that is the same as or similar to request subsystem 116, according to one or more embodiments.

At operation 1306, the subgraph may be stored in temporary data storage. Process 1306 may be performed by a request subsystem that is the same as or similar to request subsystem 116, according to one or more embodiments.

At operation 1308, a next data request that matches the predicted data request may be obtained. The next data request may be retrieved following storage of the subgraph. Process 1308 may be performed by a request subsystem that is the same as or similar to request subsystem 116, according to one or more embodiments.

In operation 1310, a subgraph may be retrieved from the temporary data storage. The subgraph may be retrieved based on the next data request matching the predicted data request. A query plan may be generated based on the next data request matching the predicted data request. The query plan may be generated to respond to the next data request. The query plan may be generated to include retrieving data from the temporary data storage based on the next data request matching the predicted data request. Operation 1310 may be performed by a request subsystem that is the same as or similar to request subsystem 116 according to one or more embodiments.

At operation 1312, the subgraph may be used to respond to the next data request. As an example, a data subset may be extracted from nodes and edges of one or more subgraphs, and the extracted data subset may be returned in response to a subsequent data request. Process 1312 may be performed by a request subsystem that is the same as or similar to request subsystem 116, according to one or more embodiments.

FIG. 14 depicts a flowchart of a method for reducing query-related resource usage in a data retrieval process in accordance with one or more embodiments. In operation 1402, a graph query may be obtained. Graph queries may be related to data requests or other requests. Graph queries may include patterns or other information. Process 1402 may be performed by an optimization subsystem that is the same as or similar to optimization subsystem 118, according to one or more embodiments.

In operation 1404, the graph query may be converted into a graph query set. Transformations may be performed based on graph data models, patterns of graph queries, or other information. A query set may include queries and query operators that link the queries. The query operators may include a first query operator that links a first query and a second query of the plurality of queries. Process 1404 may be performed by an optimization subsystem that is the same as or similar to optimization subsystem 118, according to one or more embodiments.

In some embodiments, the process 1404 includes providing the graph queries to a predictive model (e.g., neural network, machine learning model, etc.) to cause the predictive model to predict a given query set for each of the graph queries. May include. At least one of the given set of predicted queries may include predicted queries and predicted query operators that link the predicted queries. In such embodiments, the process 1404 may include, for each of the plurality of graph queries, providing the reference query set for that graph query as reference feedback to the predictive model, such that the predictive model The evaluation of a given query set predicted for the given query set may be performed. The predictive model may be updated based on an evaluation of the predictive model for a given set of predicted queries. In such embodiments, operation 1404 includes providing a graph query to a predictive model to convert the graph query into a query set, and providing the updated query set as reference feedback to the predictive model to make predictions. The method may include causing the model to evaluate the query set against the updated query set. The predictive model may be updated based on the predictive model's evaluation of the query set.

At operation 1406, satisfiability issues associated with the combination of results derived from the first query and the second query may be predicted. The prediction may be made prior to execution of the first query and the second query. In some embodiments, predicting the satisfiability problem includes determining first and second sources for obtaining results of the first and second queries; The method may include determining an incompatibility associated with the second template. The first template may be configured to convert a data representation from the first source into a graph data representation compatible with the graph database. A second template may be configured to convert a data representation from a second source into a graph data representation compatible with the graph database. Process 1406 may be performed by an optimization subsystem that is the same as or similar to optimization subsystem 118, according to one or more embodiments.

In operation 1408, the first query operator may be removed from the query set. In some embodiments, the first query operator may include a union that links the first query and the second query, or a join that links the first query and the second query. . Operator deletion may be based on predicted satisfiability problems or other information. The first query operator may be removed from the query set to update the query set such that the updated query set does not include the first query operator. In some embodiments, operations 1406 and/or 1408 include combining results derived from the subset of queries of the query set prior to executing the subset of queries of the query set based on the prediction of the satisfiability problem. It may also include anticipating other related satisfiability problems. The subset of queries may not include the first query or the second query. In such embodiments, operation 1408 removes the second query operator from the query set based on predictions of other satisfiability issues such that the updated query set includes the second query operator. The query set may include updating the query set so as not to include the query set. Process 1408 may be performed by an optimization subsystem that is the same as or similar to optimization subsystem 118, according to one or more embodiments.

At operation 1410, an updated query set that satisfies the graph query may be executed. Process 1410 may be performed by an optimization subsystem that is the same as or similar to optimization subsystem 118, according to one or more embodiments.

In some embodiments, one or more operations of method 1400 described above may include providing graph queries or corresponding query sets to a predictive model to determine one or more given optimal results for each of the corresponding query sets. The prediction model may also include causing the prediction model to predict the change. At least one of the predicted given optimizations is to remove from a given query set a given query operator that links multiple queries, or to merge multiple queries into a single query. , or removing one or more queries from a given query set. For each corresponding query set, method 1400 provides one or more reference optimizations for the corresponding query set as reference feedback to a predictive model to improve the one or more given optimizations predicted by the predictive model. may include evaluating against a reference optimization of. The predictive model may be updated based on an evaluation of the predictive model for a given predicted optimization. Method 1400 may include providing a graph query or an initial query set derived from a graph query to a predictive model to obtain one or more optimizations to the initial query set. Method 1400 may include converting a graph query into a query set by performing optimization on the initial query set. The method 1400 may include providing information to the predictive model as reference feedback to the predictive model indicating that the first query operator has been removed, thereby causing the predictive model to delete the first query operator. Optimizations may be evaluated against deletion. The predictive model may be updated based on an evaluation of the optimization by the predictive model.

In some embodiments, one or more operations of method 1400 described above provide a given query set to a predictive model to determine one or more satisfiability possibilities associated with each of the given query sets. It may also include a predictive model predicting the problem. The method 1400 includes, for each given query set, providing one or more reference satisfiability problems for the given query set as reference feedback to a predictive model to determine the satisfiability problem predicted by the predictive model. may include evaluating against a reference satisfiability problem. The predictive model may be updated based on the predictive model's evaluation of the predicted satisfiability problem. Method 1400 may include providing a query set to a predictive model to obtain information from the predictive model indicative of a prediction of the predicted satisfiability problem. Method 1400 may include predicting satisfiability problems associated with combining results derived from the first query and the second query based on the above information from the predictive model.

In some embodiments, the various computers and subsystems illustrated in FIG. 1 may include one or more computing devices programmed to perform the functions described herein. A computing device may include one or more electronic storage devices (e.g., data source 132 or other electronic storage devices), one or more physical devices programmed using one or more computer program instructions. It may also include a processor or other components. The computing device may communicate with a network (e.g., network 150) or other computing platform using wired or wireless technology (e.g., Ethernet, fiber optic, coaxial cable, WiFi, Bluetooth, near field communication, or other technology). may also include communication lines or ports that allow for the exchange of information via (e.g.) A computing device may include multiple hardware, software, or firmware components that work together. For example, a computing device may be implemented by multiple computing platforms cooperating as a computing device.

Electronic storage may include non-transitory storage media that electronically store information. The electronic storage media of electronic storage may include one or both of (i) system storage that is integral (e.g., substantially non-removable) with the server or client device, or (ii) removable storage that is removably connected to the server or client device, for example, via a port (e.g., USB port, Firewire port, etc.) or drive (e.g., disk drive, etc.). Electronic storage may include one or more of an optically readable storage medium (e.g., optical disk, etc.), a magnetically readable storage medium (e.g., magnetic tape, magnetic hard drive, floppy drive, etc.), a charge-based storage medium (e.g., EEPROM, RAM, etc.), a solid-state storage medium (e.g., flash drive, etc.), or other electronically readable storage medium. Electronic storage may include one or more virtual storage resources (e.g., cloud storage, virtual private network, or other virtual storage resources). The electronic storage may store software algorithms, information determined by a processor, information obtained from a server, information obtained from a client device, or other information that enables the functionality described herein.

A processor may be programmed to perform information processing functions in a computing device. To this end, a processor may include one or more of a digital processor, an analog processor, a digital circuit for processing information, an analog circuit for processing information, a state machine, and/or other mechanisms for electronically processing information. It may be included. In some embodiments, a processor may include multiple processing units. These processing units may be physically located within the same device, or multiple processors may perform the processing functions of multiple devices working in concert. The processor may be programmed to execute computer program instructions to implement the functions described herein of subsystems 112-120 or other subsystems. A processor is programmed to execute computer program instructions by software, hardware, firmware, combined software, hardware, or firmware in any way, or other mechanism for configuring processing functions in the processor. Good too.

It should be understood that the description of functionality provided by the different subsystems 112-120 described herein is for illustrative purposes and is not intended to be limiting. Any of the subsystems 112-120 may provide more or less functionality than those described. For example, one or more of subsystems 112-120 may be omitted, and some or all of its functionality may be provided by other subsystems 112-120. As another example, additional subsystems may be programmed to perform some or all of the functionality ascribed herein to one of subsystems 112-120.

Although the invention has been described in detail for purposes of illustration and in accordance with the embodiments presently considered most practical and preferred, it is to be understood that such detailed description is for purposes of illustration only. The invention is not intended to be limited to the disclosed embodiments, but on the contrary, it is intended that the invention cover modifications and equivalent constructions falling within the spirit and scope of the appended claims. For example, it is to be understood that the invention may combine one or more features of any embodiment with one or more features of any other embodiments to the extent possible.

The present technology can be better understood by referring to the embodiments listed below.
[Embodiment 1]
first modeling information related to a first graph data model, the first data representation being incompatible with the first graph database to a graph data representation compatible with the first graph database; obtaining the first modeling information including a first template for conversion;
providing one or more of the first templates and the first data representation to the machine learning model;
The machine learning model is configured to predict one or more additional templates of the first template independently of the one or more additional templates of the first template, and the method further comprises: , providing the one or more additional templates of the first template to the machine learning model as reference feedback for predictions of the one or more additional templates by the machine learning model to train the machine learning model; process and
A given graph data model generates one or more templates for converting a data representation of a given database into a graph data representation of a given graph database. providing a collection to the machine learning model.
[Embodiment 2]
2. The method of embodiment 1, wherein the graph data representation of the given graph database includes graph nodes or edges connecting the graph nodes.
[Embodiment 3]
second modeling information associated with a second graph data model, the second data representation being incompatible with the second graph database to a graph data representation compatible with the second graph database; obtaining the second modeling information including a second template for conversion;
providing one or more of the second templates and the second data representation to the machine learning model;
the machine learning model is configured to predict one or more additional templates of the second template independently of the one or more additional templates of the second template;
The one or more additional templates of the second template are used as reference feedback for the prediction of the one or more additional templates of the second template by the machine learning model to train the machine learning model. 3. The method of embodiment 1 or 2, comprising providing a learning model.
[Embodiment 4]
4. The method of embodiment 3, wherein the first data representation and the second data representation are compatible with the same database.
[Embodiment 5]
The first data representation is compatible with a first non-graph database, and the second data representation is compatible with a second non-graph database but not compatible with the first non-graph database. The method of embodiment 3, wherein no.
[Embodiment 6]
4. The method of embodiment 3, wherein the first data representation or the second data representation is compatible with at least one graph database.
[Embodiment 7]
Graph information relating to nodes and edges of graphs in the given graph database and indicating a plurality of data representation sets, each of the data representation sets of the plurality of data representation sets including nodes and edges connecting the nodes. , obtaining the graph information;
For each data representation set of the plurality of data representation sets, the machine learning model predicts one or more nodes or edges of the data representation set. and providing the machine learning model with
The machine learning model is configured to perform predictions of the one or more additional nodes or edges without relying on the one or more additional nodes or edges to train the machine learning model. for each data representation set of the plurality of data representation sets, the one or more additional nodes of the data representation set as reference feedback for the prediction of the one or more additional nodes or edges by the machine learning model. or providing additional edges to the machine learning model.
[Embodiment 8]
performing a traversal of the graph to process nodes or edges of the graph through the machine learning model, causing the machine learning model to generate new nodes or new edges of the graph;
8. The method of embodiment 7, further comprising adding the new node or the new edge from the machine learning model to the graph.
[Embodiment 9]
retrieving a given node or edge from the machine learning model based on the traversal of the graph;
adding the given node or edge from the machine learning model to the graph based on user approval for adding the given node or edge;
9. The method of embodiment 8, further comprising providing information indicative of the user's consent to the machine learning model as reference feedback regarding the generation of the given node or edge by the machine learning model.
[Embodiment 10]
retrieving a given node or edge from the machine learning model based on the traversal of the graph;
and obtaining a user's refusal to add the given node or edge,
the given node or edge is not added to the graph based on the user's refusal;
9. The method of embodiment 8, the method further comprising providing information indicative of the user's refusal to the machine learning model as reference feedback regarding the generation of the predetermined node or edge by the machine learning model. .
[Embodiment 11]
a step of predicting that a data request will occur in the future;
generating, in response to the prediction of the data request, one or more subgraphs representing a data subset of the data set that is the data that the data request is predicted to seek based on a graph data model;
storing the one or more subgraphs in temporary data storage;
retrieving a next data request that matches the predicted data request after the storage of the one or more subgraphs;
retrieving the one or more subgraphs from the primary data storage based on the next data request matching the predicted data request; and retrieving the one or more subgraphs in response to the next data request. A process using and a method comprising.
[Embodiment 12]
The step of using the one or more subgraphs includes:
extracting the data subset from nodes and edges of the one or more subgraphs;
12. The method of embodiment 11, comprising returning the extracted subset of data in response to the next data request.
[Embodiment 13]
further comprising retrieving a first data subset of the data subsets from one or more non-graph databases in response to the prediction of the data request;
The step of generating one or more subgraphs includes, after obtaining the first data subset, generating a first subgraph representing the first data subset based on the graph data model,
13. The method of embodiment 11 or 12, wherein the one or more subgraphs include the first subgraph to use the first subgraph to respond to the next data request.
[Embodiment 14]
further comprising retrieving a second data subset of the data subsets from one or more graph databases in response to the prediction of the data request;
14. The method of embodiment 13, wherein the one or more subgraphs include the second subgraph representing the second data subset, such as using the second subgraph to respond to the next data request. .
[Embodiment 15]
generating a plurality of graph queries based on one or more search parameters of the predicted data request;
converting at least one of the plurality of graph queries into one or more non-graph queries based on the graph data model;
executing the one or more non-graph queries to obtain the first data subset from the one or more non-graph databases;
15. The method of embodiment 14, further comprising executing another one of the plurality of graph queries to obtain the second data subset from the one or more non-graph databases. .
[Embodiment 16]
further comprising generating a query plan responsive to the next data request based on the next data request that matches the predicted data request;
the query plan is generated to include retrieving data from the temporary data storage based on the next data request matching the predicted data request;
16. The method as in any one of embodiments 11-15, wherein retrieving the one or more subgraphs includes retrieving the one or more subgraphs from the temporary data storage based on the query plan. .
[Embodiment 17]
further comprising generating a query plan responsive to the next data request based on the next data request that matches the predicted data request;
the query plan is generated in response to the next data request;
The step of retrieving the one or more subgraphs includes retrieving the one or more subgraphs from the temporary data storage and retrieving one or more other data subsets from the one or more other data based on the query plan. including obtaining from a source;
Using the one or more subgraphs includes (i) the data subset represented by the one or more subgraphs, and (ii) the one or more other 17. The method as in any one of embodiments 11-16, comprising using the data.
[Embodiment 18]
further comprising executing a query in response to the prediction of the data request;
the executed query is part of a set of queries that would have been executed in response to the predicted data request if the predicted data request had been obtained from a client device;
further comprising, based on the query, obtaining the data subset of the data set that the data request is expected to require;
18. The method as in any one of embodiments 11-17, wherein generating the one or more subgraphs includes generating the one or more subgraphs based on the data subset and the graph data model. .
[Embodiment 19]
19. The method of embodiment 18, wherein execution of one or more other queries of the set of queries does not occur from the prediction of the data request.
[Embodiment 20]
providing past queries compatible with the graph data model to the machine learning model to train the machine learning model;
after the training of the machine learning model, obtaining information from the machine learning model indicating the prediction of the data request;
providing information indicative of the next data request as reference feedback to the machine learning model based on the next data request matching the predicted data request to further train the machine learning model; 20. The method as in any one of embodiments 11-19, further comprising.
[Embodiment 21]
obtaining the graph query that is related to a data request and that includes a plurality of patterns;
converting the graph query into a query set based on the graph data model and the plurality of patterns of the graph query,
The query set includes a plurality of queries and a query operator that links the plurality of queries, and the query operator includes a first query that links a first query and a second query among the plurality of queries. contains query operators,
The method further includes:
predicting satisfiability problems associated with combining results derived from the first query and the second query before the first query and the second query are executed;
performing one or more optimizations of the query set based on the prediction of the satisfiability problem to update the query set;
executing the updated query set to satisfy the graph query.
[Embodiment 22]
The step of performing the one or more optimizations includes removing the first query operator from the query set based on the prediction of the satisfiability problem so that the updated query set 22. The method of embodiment 21, comprising updating the query set to no longer include one query operator.
[Embodiment 23]
prior to executing a subset of the plurality of queries of the query set, based on the prediction of the satisfiability problem, from a sub-subset of the queries that does not include the first query or the second query. predicting alternative satisfiability problems associated with combining the results obtained;
removing the second query operator from the query set based on the prediction of the another satisfiability problem such that the updated query set does not include the second query operator; 23. The method of embodiment 22, further comprising updating the query set.
[Embodiment 24]
Embodiment 21, wherein the first query operator includes a union that links the first query and the second query, or a join that links the first query and the second query. The method described in any one of 23 to 23.
[Embodiment 25]
The step of predicting the satisfiability problem includes:
determining a first source and a second source for obtaining results for the first query and the second query;
determining incompatibility regarding the first template and the second template;
The first template is configured to convert a data representation from the first source into a graph data representation that is compatible with a graph database, and the second template is configured to convert a data representation from the second source to a graph data representation that is compatible with a graph database. configured to convert a data representation into a graph data representation compatible with the graph database;
25. The method as in any one of embodiments 21-24, the method further comprising predicting the satisfiability problem based on the incompatibility with the first template and the second template. .
[Embodiment 26]
The step of predicting the satisfiability problem includes:
determining a first data type as a data type for storing a first result for the first query in a graph database based on the graph data model;
determining a second data type as a data type for storing a second result for the second query in the graph database based on the graph data model;
determining an incompatibility associated with the first data type and the second data type; and determining the satisfiability based on the incompatibility associated with the first data type and the second data type. 26. The method as in any one of embodiments 21-25, comprising predicting sexual problems. [Embodiment 27]
a first data representation corresponding to said first result as a data type compatible with a data type used to store said second data representation corresponding to said second result in a second source; 27. The method of embodiment 26, wherein the is stored in the first source.
[Embodiment 28]
a first data representation corresponding to said first result as a data type that is incompatible with a data type used to store said second data representation corresponding to said second result in a second source; 27. The method of embodiment 26, wherein the is stored in the first source.
[Embodiment 29]
further comprising providing the plurality of graph queries to the machine learning model to cause the machine learning model to predict a given query set for each of the plurality of graph queries;
at least one of the given set of predicted queries includes a predicted query and a predicted query operator linking the predicted queries;
The method further includes, for each of the plurality of graph queries, providing a reference query set for the graph query as reference feedback to the machine learning model, causing the machine learning model to comprising a step of performing an evaluation on a reference query set,
the machine learning model is updated based on the evaluation made by the machine learning model of the given set of predicted queries;
The method further includes:
converting the graph query into the query set by providing the graph query to the machine learning model to obtain the query set;
providing the updated query set to the machine learning model as reference feedback to the machine learning model to cause the machine learning model to evaluate the query set against the updated query set; Equipped with
29. The method as in any one of embodiments 21-28, wherein the machine learning model is updated based on the evaluation of the query set performed by the machine learning model.
[Embodiment 30]
providing a plurality of graph queries or a plurality of corresponding query sets to a machine learning model to cause the machine learning model to predict one or more given optimizations for each of the plurality of corresponding query sets; Further prepare,
At least one of the predicted given optimizations includes removing from a given query set a given query operator that links multiple queries, merging multiple queries into one query. or removing one or more queries from a given query set;
The method includes, for each of the plurality of corresponding query sets, providing one or more reference optimizations for the corresponding query set as reference feedback to the machine learning model to cause the machine learning model to further comprising the step of evaluating the one or more predicted given optimizations against the one or more reference optimizations;
the machine learning model is updated based on the evaluation of the one or more predicted given optimizations performed by the machine learning model;
The method further includes:
providing the graph query or an initial query set derived from the graph query to the machine learning model to obtain one or more optimizations to the initial query set;
converting the graph query into the query set by performing the one or more optimizations on the initial query set;
providing information to the machine learning model indicating that the first query operator has been removed as reference feedback to the machine learning model; and a step of evaluating whether the query operator has been deleted,
30. The method as in any one of embodiments 21-29, wherein the machine learning model is updated based on the evaluation of the one or more optimizations performed by the machine learning model.
[Embodiment 31]
providing a given plurality of query sets to a machine learning model to cause the machine learning model to predict one or more satisfiability problems associated with each of the given plurality of query sets;
For each of the given plurality of query sets, providing one or more reference satisfiability problems for the given query set to the machine learning model as reference feedback to the machine learning model, further comprising causing the model to evaluate the one or more predicted satisfiability problems against the one or more reference satisfiability problems;
the machine learning model is updated based on the evaluation of the one or more predicted satisfiability problems performed by the machine learning model;
The method further includes:
providing the query set to the machine learning model to obtain information from the machine learning model indicating the prediction of the satisfiability problem;
predicting the satisfiability problem associated with combining results derived from the first query and the second query based on the information from the machine learning model. The method according to any one of Forms 21 to 30.
[Embodiment 32]
A tangible, non-transitory, machine-readable medium storing instructions that, when executed by a data processing device, cause the data processing device to perform a process comprising any of embodiments 1-31. , machine-readable medium.
[Embodiment 33]
A system comprising one or more processors and a memory storing instructions, wherein when the instructions are executed by the one or more processors, the one or more processors - A system in which processing including any of 31 is realized.

Claims (60)

A system for reducing query-related resource usage in a data retrieval process, the system comprising:
a computer system having one or more processors programmed using computer program instructions; when the computer program instructions are executed, the computer system:
Obtaining a graph query related to a data request, the graph query including a plurality of patterns;
converting the graph query into a query set based on a graph data model and the plurality of patterns of the graph query;
The query set includes a plurality of queries and a query operator that links the plurality of queries;
The query operator includes a first query operator that links a first query and a second query among the plurality of queries,
predicting satisfiability problems associated with combining results derived from the first query and the second query before the first query and the second query are executed;
Based on the prediction of the satisfiability problem, the first query operator is removed from the query set such that the updated query set does not include the first query operator. update the set,
A method of executing the updated query set to satisfy the graph query. The first query operator includes a union that links the first query and the second query, or a join that links the first query and the second query. system described in. Predicting the satisfiability problem is:
determining a first source and a second source for obtaining results for the first query and the second query;
determining an incompatibility regarding the first template and the second template; and
predicting the satisfiability problem based on the incompatibility with the first template and the second template;
The first template is configured to convert a data representation from the first source into a graph data representation that is compatible with a graph database, and the second template is configured to convert a data representation from the second source to a graph data representation that is compatible with a graph database. 2. The system of claim 1, wherein the system is configured to convert a data representation to a graph data representation compatible with the graph database. Predicting the satisfiability problem
determining a first data type based on the graph data model as a data type for storing a first result to the first query in a graph database;
determining a second data type based on the graph data model as a data type for storing a second result to the second query in the graph database;
determining an incompatibility associated with the first data type and the second data type; and
2. The system of claim 1, further comprising: predicting the satisfiability problem based on the incompatibility related to the first data type and the second data type.
a first data representation corresponding to said first result as a data type compatible with a data type used to store said second data representation corresponding to said second result in a second source; 5. The system of claim 4, wherein: is stored in the first source. a first data representation corresponding to said first result as a data type that is incompatible with a data type used to store said second data representation corresponding to said second result in a second source; 5. The system of claim 4, wherein: is stored in the first source. The computer system further includes:
prior to executing a subset of the plurality of queries of the query set, based on the prediction of the satisfiability problem, from a sub-subset of the queries that does not include the first query or the second query. Anticipate other satisfiability problems associated with combining the results of
removing the second query operator from the query set based on the prediction of the another satisfiability problem such that the updated query set does not include the second query operator; The system of claim 1, updating the query set. The computer system further includes:
providing a plurality of graph queries to a neural network, causing the neural network to predict a given query set for each of the plurality of graph queries, at least one of the predicted given query sets comprising: a predicted query and a predicted query operator linking the predicted query;
For each of the plurality of graph queries, providing a reference query set for the graph query as reference feedback to the neural network to cause the neural network to apply the predicted given query set to the reference query set. and updating the neural network based on the neural network's evaluation of the predicted given query set;
converting the graph query into the query set by providing the graph query to the neural network to obtain the query set;
providing the updated query set to the neural network as reference feedback to the neural network, causing the neural network to evaluate the query set against the updated query set; 2. The system of claim 1, wherein the neural network is updated based on the evaluation by. The computer system further includes:
providing a plurality of graph queries or a plurality of corresponding query sets to a neural network and causing the neural network to predict one or more given optimizations for each of the plurality of corresponding query sets;
At least one of the predicted given optimizations includes removing a given query operator that links multiple queries from a given query set, merging multiple queries into a single query. or removing one or more queries from a given query set;
For each of the plurality of corresponding query sets, one or more reference optimizations for the corresponding query set are provided as reference feedback to the neural network to cause the neural network to calculate the one or more predicted given the neural network is updated based on the neural network's evaluation of the predicted given query set;
providing the graph query or an initial query set derived from the graph query to the neural network to obtain one or more optimizations for the initial query set;
converting the graph query into the query set by performing the one or more optimizations on the initial query set;
providing information to the neural network indicating that the first query operator has been removed as reference feedback to the neural network; and having the neural network perform the one or more optimizations on the first query operator. 2. The system of claim 1, wherein the neural network is updated based on the neural network's evaluation of the one or more optimizations. The computer system further includes:
providing a given plurality of query sets to a neural network to cause the neural network to predict one or more satisfiability problems associated with each of the given plurality of query sets;
providing one or more reference satisfiability problems for the given query set to the neural network as reference feedback to the neural network for each of the given plurality of query sets; , causing the one or more predicted satisfiability problems to be evaluated against the one or more reference satisfiability problems, and evaluating the one or more predicted satisfiability problems by the neural network. the neural network is updated based on
providing the query set to the neural network to obtain information from the neural network indicating the prediction of the satisfiability problem;
The system of claim 1, wherein the system predicts the satisfiability problem associated with combining results derived from the first query and the second query based on the information from the neural network. . A method of reducing query-related resource usage in a data retrieval process, the method comprising: a computer system comprising one or more processors executing computer program instructions; , carrying out the method;
The method includes:
obtaining the graph query, which is a query related to a data request and includes a plurality of patterns;
converting the graph query into a query set based on the graph data model and the plurality of patterns of the graph query,
The query set includes a plurality of queries and a query operator that links the plurality of queries, and the query operator includes a first query that links a first query and a second query among the plurality of queries. contains query operators,
The method further includes:
predicting satisfiability problems associated with combining results derived from the first query and the second query before the first query and the second query are executed;
Based on the prediction of the satisfiability problem, the first query operator is removed from the query set such that the updated query set does not include the first query operator. a step of updating the set;
executing the updated query set to satisfy the graph query. The first query operator includes a union that links the first query and the second query, or a join that links the first query and the second query.
The method according to claim 11. The step of predicting the satisfiability problem includes:
determining a first source and a second source for obtaining results for the first query and the second query;
determining an incompatibility with a first template and a second template; and predicting the satisfiability problem based on the incompatibility with the first template and the second template; including;
The first template is configured to convert a data representation from the first source into a graph data representation compatible with a graph database, and the second template is configured to convert a data representation from the second source to a graph data representation that is compatible with a graph database. is configured to convert a data representation into a graph data representation compatible with the graph database;
12. The method of claim 11, further comprising predicting the satisfiability problem based on the incompatibility with the first template and the second template. The step of predicting the satisfiability problem includes:
determining a first data type as a data type for storing a first result for the first query in a graph database based on the graph data model;
determining a second data type as a data type for storing a second result for the second query in the graph database based on the graph data model;
determining an incompatibility associated with the first data type and the second data type; and
12. The method of claim 11, comprising predicting the satisfiability problem based on the incompatibility associated with the first data type and the second data type. a first data representation corresponding to said first result as a data type compatible with a data type used to store said second data representation corresponding to said second result in a second source; 15. The method of claim 14, wherein: is stored in the first source. a first data representation corresponding to said first result as a data type that is incompatible with a data type used to store said second data representation corresponding to said second result in a second source; 15. The method of claim 14, wherein: is stored in the first source. prior to executing a subset of the plurality of queries of the query set, based on the prediction of the satisfiability problem, from a sub-subset of the queries that does not include the first query or the second query. predicting alternative satisfiability problems associated with combining the results obtained;
removing the second query operator from the query set based on the prediction of the another satisfiability problem such that the updated query set does not include the second query operator; 12. The method of claim 11, further comprising updating the query set. further comprising providing a plurality of graph queries to a neural network and causing the neural network to predict a given query set for each of the plurality of graph queries;
at least one of the given set of predicted queries includes a predicted query and a predicted query operator linking the predicted queries;
The method further includes, for each of the plurality of graph queries, providing a reference query set for the graph query as reference feedback to the neural network to cause the neural network to use the predicted given query set as reference feedback. Equipped with a process to evaluate the query set,
the neural network is updated based on the neural network's evaluation of the predicted given query set;
The method further includes:
converting the graph query into the query set by providing the graph query to the neural network to obtain the query set;
providing the updated query set to the neural network as reference feedback to the neural network, causing the neural network to evaluate the query set against the updated query set;
12. The method of claim 11, wherein the neural network is updated based on the neural network's evaluation of the query set. providing a plurality of graph queries or a plurality of corresponding query sets to a neural network to cause the neural network to predict one or more given optimizations for each of the plurality of corresponding query sets;
At least one of the predicted given optimizations includes removing a given query operator that links multiple queries from a given query set, merging multiple queries into a single query. or removing one or more queries from a given query set;
The method further includes, for each of the plurality of corresponding query sets, providing one or more reference optimizations for the corresponding query set as reference feedback to the neural network to cause the neural network to calculate the predicted result. evaluating one or more given optimizations against the one or more reference optimizations;
the neural network is updated based on the neural network's evaluation of the predicted one or more given optimizations;
The method further includes:
providing the graph query or an initial query set derived from the graph query to the neural network to obtain one or more optimizations for the initial query set;
converting the graph query into the query set by performing the one or more optimizations on the initial query set;
providing information to the neural network indicating that the first query operator has been removed as reference feedback to the neural network; and having the neural network perform the one or more optimizations on the first query operator. a step of evaluating whether the has been deleted;
12. The method of claim 11, wherein the neural network is updated based on the neural network's evaluation of the one or more optimizations. providing a given plurality of query sets to a neural network to cause the neural network to predict one or more satisfiability problems associated with each of the given plurality of query sets;
providing one or more reference satisfiability problems for the given query set to the neural network as reference feedback to the neural network for each of the given plurality of query sets; evaluating the one or more predicted satisfiability problems against the one or more reference satisfiability problems,
updating the neural network based on the neural network's evaluation of the predicted one or more satisfiability problems;
The method further includes:
providing the query set to the neural network to obtain information from the neural network indicating the prediction of the satisfiability problem;
predicting the satisfiability problem associated with combining results derived from the first query and the second query based on the information from the neural network. The method described in. A system for reducing data retrieval delay through prediction-based data subgraph generation, the system comprising:
comprising a computer system having one or more processors programmed using computer program instructions;
When the computer program instructions are executed, the computer system:
Predict future data requests based on past queries compatible with graph data models,
generating, in response to the prediction of the data request, one or more subgraphs representing a data subset of the data set that is the data that the data request is predicted to seek based on a graph data model;
storing the one or more subgraphs in temporary data storage;
after the storing of the one or more subgraphs, obtaining a next data request that matches the predicted data request;
retrieving the one or more subgraphs from the primary data storage based on the next data request that matches the predicted data request;
A system that uses the one or more subgraphs to respond to the next data request. The computer system includes:
extracting the data subset from nodes and edges of the one or more subgraphs;
22. The system of claim 21, using the one or more subgraphs to respond to the next data request by returning the extracted subset of data. The computer system further includes:
in response to the prediction of the data request, retrieving a first data subset of the data subsets from one or more non-graph databases;
after obtaining the first data subset, generating a first subgraph representing the first data subset based on the graph data model;
22. The system of claim 21, wherein the one or more subgraphs include the first subgraph to use the first subgraph to respond to the next data request. The computer system further obtains a second data subset of the data subsets from one or more graph databases in response to the prediction of the data request;
24. The system of claim 23, wherein the one or more subgraphs include the second subgraph representing the second data subset to use the second subgraph to respond to the next data request. . The computer system further includes:
generating a plurality of graph queries based on one or more search parameters of the predicted data request;
converting at least one of the plurality of graph queries into one or more non-graph queries based on the graph data model;
executing the one or more non-graph queries to obtain the first data subset from the one or more non-graph databases;
25. The system of claim 24, executing another one of the plurality of graph queries to obtain the second data subset from the one or more non-graph databases. The computer system further generates a query plan to respond to the next data request based on the next data request that matches the predicted data request;
the query plan is generated including retrieving data from the temporary data storage based on the next data request matching the predicted data request;
22. The system of claim 21, wherein the computer system further retrieves the one or more subgraphs from the temporary data storage based on the query plan. The computer system further generates a query plan responsive to the next data request based on the next data request that matches the predicted data request;
the query plan is generated in response to the next data request;
The computer system further includes:
retrieving the one or more subgraphs from the temporary data storage and retrieving one or more other data subsets from one or more other data sources based on the query plan;
22. The system of claim 21, using (i) the data subset represented by the one or more subgraphs, and (ii) the one or more other data to respond to the next data request. . The computer system further includes:
executing a query in response to the prediction of the data request;
the executed query is part of a set of queries that would have been executed in response to the predicted data request if the predicted data request had been obtained from a client device;
The computer system further includes:
obtaining the data subset of the data set that the data request is expected to require based on the query;
22. The system of claim 21, generating the one or more subgraphs based on the data subset and the graph data model. 29. The system of claim 28, wherein no execution of one or more other queries of the set of queries occurs from the prediction of the data request. The computer system further includes:
providing past queries compatible with the graph data model to the machine learning model to train the machine learning model;
after the training of the machine learning model, obtaining information from the machine learning model indicating the prediction of the data request;
4. Provide information indicative of the next data request as reference feedback to the machine learning model based on the next data request matching the predicted data request to further train the machine learning model. The system described in 21. A method of reducing data retrieval delay through prediction-based generation of data subgraphs, the method comprising: a computer system comprising one or more processors executing computer program instructions; executes the method, and the method includes:
predicting future data requests based on past queries compatible with the graph data model;
generating, in response to the prediction of the data request, one or more subgraphs representing a data subset of the data set that is the data that the data request is predicted to seek based on a graph data model;
storing the one or more subgraphs in temporary data storage;
retrieving a next data request that matches the predicted data request after the storage of the one or more subgraphs;
retrieving the one or more subgraphs from the primary data storage based on the next data request that matches the predicted data request;
using the one or more subgraphs to respond to the next data request. The step of using the one or more subgraphs includes:
extracting the data subset from nodes and edges of the one or more subgraphs;
32. The method of claim 31, comprising returning the extracted subset of data in response to the next data request. further comprising retrieving a first data subset of the data subsets from one or more non-graph databases in response to the prediction of the data request;
The step of generating one or more subgraphs includes, after obtaining the first data subset, generating a first subgraph representing the first data subset based on the graph data model;
32. The method of claim 31, wherein the one or more subgraphs include the first subgraph to use the first subgraph to respond to the next data request. further comprising retrieving a second data subset of the data subsets from one or more graph databases in response to the prediction of the data request;
34. The method of claim 33, wherein the one or more subgraphs include the second subgraph representing the second data subset, such as using the second subgraph to respond to the next data request. . generating a plurality of graph queries based on one or more search parameters of the predicted data request;
converting at least one of the plurality of graph queries into one or more non-graph queries based on the graph data model;
executing the one or more non-graph queries to obtain the first data subset from the one or more non-graph databases;
35. The method of claim 34, further comprising executing another one of the plurality of graph queries to obtain the second data subset from the one or more non-graph databases. . further comprising generating a query plan responsive to the next data request based on the next data request that matches the predicted data request;
the query plan is generated to include retrieving data from the temporary data storage based on the next data request matching the predicted data request;
32. The method of claim 31, wherein retrieving the one or more subgraphs includes retrieving the one or more subgraphs from the temporary data storage based on the query plan. further comprising generating a query plan responsive to the next data request based on the next data request that matches the predicted data request;
the query plan is generated in response to the next data request;
The step of retrieving the one or more subgraphs includes retrieving the one or more subgraphs from the temporary data storage based on the query plan and retrieving one or more other data subsets from one or more other data. including obtaining from a source;
Using the one or more subgraphs includes (i) the data subset represented by the one or more subgraphs, and (ii) the one or more other 32. The method of claim 31, comprising using data. further comprising executing a query in response to the prediction of the data request;
the executed query is part of a set of queries that would have been executed in response to the predicted data request if the predicted data request had been obtained from a client device;
The method further comprises obtaining the data subset of the data set that the data request is expected to require based on the query;
32. The method of claim 31, wherein generating the one or more subgraphs includes generating the one or more subgraphs based on the data subset and the graph data model. 39. The method of claim 38, wherein no execution of one or more other queries of the set of queries occurs from the prediction of the data request.
a computer system having one or more processors configured to
When the computer program instructions are executed, the computer system:
obtaining modeling information associated with a graph data model set, the modeling information associated with each graph data model of the graph data model set converting a non-graph data representation in a non-graph database into a graph data compatible with the graph database; Contains templates for converting into expressions,
one or more of the templates of the graph data model and the non-graph data representation for each of the graph data models of the set of graph data models and the non-graph data representation that the graph data model converts; a neural network to cause the neural network to predict one or more additional templates of the templates of the graph data model; configured to perform the prediction of the additional templates,
providing the neural network with the one or more additional templates of the templates of the graph data models for each graph data model of the graph data model set to train the neural network;
providing a collection of non-graph data representations from a given non-graph database to the neural network to convert the non-graph data representation in the given non-graph database into a graph data representation compatible with the given graph database; a system that causes the neural network to generate one or more templates for a given graph data model for converting to a graph data model. providing past queries compatible with the graph data model to the machine learning model to train the machine learning model;
after the training of the machine learning model, obtaining information from the machine learning model indicating the prediction of the data request;
providing information indicative of the next data request as reference feedback to the machine learning model based on the next data request matching the predicted data request to further train the machine learning model; 32. The method of claim 31, further comprising: A system that provides neural network-based generation of graph data models for converting non-graph data representations to compatible graph data representations, the system comprising:
comprising a computer system having one or more processors programmed using computer program instructions;
When the computer program instructions are executed, the computer system:
obtaining modeling information associated with a graph data model set, the modeling information associated with each graph data model of the graph data model set converting a non-graph data representation in a non-graph database into a graph data compatible with the graph database; Contains templates for converting into expressions,
one or more of the templates of the graph data model and the non-graph data representation for each of the graph data models of the set of graph data models and the non-graph data representation that the graph data model converts; a neural network to cause the neural network to predict one or more additional templates of the templates of the graph data model; configured to perform the prediction of the additional templates,
providing the neural network with the one or more additional templates of the templates of the graph data models for each graph data model of the graph data model set to train the neural network;
providing a collection of non-graph data representations from a given non-graph database to the neural network to convert the non-graph data representation in the given non-graph database into a graph data representation compatible with the given graph database; a system that causes the neural network to generate one or more templates for a given graph data model for converting to a graph data model. 42. The system of claim 41, wherein the graph data representation compatible with the given graph database includes graph nodes or edges connecting the graph nodes. The computer system further includes:
obtaining graph information associated with graph nodes and edges in the given graph database and indicating a plurality of data representation sets;
Each data representation set of the plurality of data representation sets includes nodes and edges connecting the nodes;
For each data representation set of the plurality of data representation sets, the neural network predicts one or more nodes or edges of the data representation set to predict one or more additional nodes or edges of the data representation set. providing to the neural network;
the neural network is configured to perform predictions of the one or more additional nodes or edges independently of the one or more additional nodes or edges;
To train the neural network, for each data representation set of the plurality of data representation sets, the one or more additional nodes or edges of the data representation set are trained by the neural network. 42. The system of claim 41, providing reference feedback to the neural network for predicting additional edges. The computer system further includes:
performing a traversal of the graph to process nodes or edges of the graph via the neural network, causing the neural network to generate new nodes or new edges for the graph;
44. The system of claim 43, adding at least one of the new node or the new edge from the neural network to the graph. The computer system further includes:
obtaining a given node or edge from the neural network based on the traversal of the graph;
adding the given node or edge from the neural network to the graph based on user consent for adding the given node or edge;
45. The system of claim 44, wherein information indicative of the user's acceptance is provided to the neural network as reference feedback regarding the generation of the given node or edge by the neural network. The computer system further includes:
obtaining a given node or edge from the neural network based on the traversal of the graph;
obtaining a user's refusal to add the given node or edge, wherein the given node or edge is not added to the graph based on the user's refusal;
45. The system of claim 44, wherein information indicative of the user's acceptance is provided to the neural network as reference feedback regarding the generation of the given node or edge by the neural network. 42. The system of claim 41, wherein each graph data model of the set of graph data models is configured to convert a database row into a graph data representation. A method of providing machine learning model-based generation of a graph data model for converting a non-graph data representation into a graph data representation, the method comprising one or more processors executing computer program instructions. implemented by a computer system, when the computer program instructions are executed, perform the method;
The method includes:
first modeling information related to a first graph data model, the first data representation being incompatible with the first graph database to a graph data representation compatible with the first graph database; obtaining the first modeling information including a first template for conversion;
providing one or more of the first templates and the first data representation to the machine learning model;
the machine learning model is configured to predict one or more additional templates of the first template independently of the one or more additional templates of the first template;
The method further includes:
providing the one or more additional templates of the first template to the machine learning model as reference feedback for predictions of the one or more additional templates by the machine learning model to train the machine learning model; and,
A given graph data model generates one or more templates for converting a data representation of a given database into a graph data representation of a given graph database. providing a collection to the machine learning model. 49. The method of claim 48, wherein the graph data representation of the given graph database includes graph nodes or edges connecting the graph nodes. second modeling information associated with a second graph data model, the second data representation being incompatible with the second graph database to a graph data representation compatible with the second graph database; obtaining the second modeling information including a second template for conversion;
providing one or more of the second templates and the second data representation to the machine learning model;
the machine learning model is configured to predict one or more additional templates of the second template independently of the one or more additional templates of the second template;
The method further includes using the one or more additional templates of the second template as reference feedback for predictions of the one or more additional templates of the second template by the machine learning model to train the machine learning model. 49. The method of claim 48, comprising providing additional templates to the machine learning model. 51. The method of claim 50, wherein the first data representation and the second data representation are compatible with the same database. The first data representation is compatible with a first non-graph database, and the second data representation is compatible with a second non-graph database but not compatible with the first non-graph database. 51. The method of claim 50, wherein no. 51. The method of claim 50, wherein the first data representation or the second data representation is compatible with at least one graph database. Graph information relating to nodes and edges of graphs in the given graph database and indicating a plurality of data representation sets, each of the data representation sets of the plurality of data representation sets including nodes and edges connecting the nodes. , obtaining the graph information;
For each data representation set of said plurality of data representation sets, said machine learning model predicts one or more additional nodes or edges of said data representation set. further comprising the step of providing the machine learning model with
the machine learning model is configured to perform predictions of the one or more additional nodes or edges independently of the one or more additional nodes or edges;
The method further includes, for each data representation set of the plurality of data representation sets, as reference feedback for the prediction of the one or more additional nodes or edges by the machine learning model to train the machine learning model. 49. The method of claim 48, comprising providing the one or more additional nodes or edges of a data representation set to the machine learning model.
performing a traversal of the graph to process nodes or edges of the graph through the machine learning model, causing the machine learning model to generate new nodes or new edges of the graph;
55. The method of claim 54, further comprising adding the new node or the new edge from the machine learning model to the graph. retrieving a given node or edge from the machine learning model based on the traversal of the graph;
adding the given node or edge from the machine learning model to the graph based on user approval for adding the given node or edge;
56. The method of claim 55, further comprising providing information indicative of the user's consent to the machine learning model as reference feedback regarding the generation of the given node or edge by the machine learning model. retrieving a given node or edge from the machine learning model based on the traversal of the graph;
and obtaining a user's refusal to add the given node or edge,
the given node or edge is not added to the graph based on the user's refusal;
56. The method further comprises providing information indicative of the user's refusal to the machine learning model as reference feedback regarding the generation of the given node or edge by the machine learning model. Method. 1. A system providing machine learning model based generation of graph data models for transforming a non-graph data representation into a compatible graph data representation, comprising:
a computer system having one or more processors programmed with computer program instructions, the computer system, when executed,
acquiring first modeling information related to a first graph data model, the first modeling information including a first template for converting a first data representation incompatible with a first graph database into a graph data representation compatible with the first graph database;
providing one or more of the first templates and the first data representation to the machine learning model;
the machine learning model is configured to predict one or more additional templates of the first template without reliance on the one or more additional templates of the first template;
providing the one or more additional templates of the first template to the machine learning model as reference feedback for predictions of the one or more additional templates by the machine learning model to train the machine learning model;
1. A system comprising: a machine learning model that provides a collection of data representations from a given database to the machine learning model, such that the machine learning model generates one or more templates for transforming the data representations of the given database into graph data representations of the given graph database. The computer system further includes:
obtaining graph information associated with graph nodes and edges in the given graph database and indicating a plurality of data representation sets;
Each data representation set of the plurality of data representation sets includes nodes and edges connecting the nodes;
For each data representation set of said plurality of data representation sets, said machine learning model predicts one or more additional nodes or edges of said data representation set. to the machine learning model;
the machine learning model is configured to perform predictions of the one or more additional nodes or edges independently of the one or more additional nodes or edges;
To train the machine learning model, for each data representation set of the plurality of data representation sets, one of the data representation sets of the plurality of data representation sets is used as reference feedback for the prediction of the one or more additional nodes or edges by the machine learning model. 59. The system of claim 58, providing one or more additional nodes or edges to the machine learning model. The computer system further includes:
performing a traversal of the graph to process nodes or edges of the graph via the machine learning model, causing the machine learning model to generate new nodes or new edges of the graph;
60. The system of claim 59, adding the new node or the new edge from the machine learning model to the graph.