JP7365446B2 - Method and system for performing reuse analysis for model lifecycle management - Google Patents

Description

TECHNICAL FIELD The present subject matter relates generally to software reuse, and more particularly, but not exclusively, to methods and systems for performing reuse analysis for model lifecycle management.

In today's world, data infrastructure is evolving to provide cross-cutting services. Data engineers often create reusable analysis code for use in data analysis to reduce the time and resources involved in developing software code, as well as the costs involved in managing newly developed software. requires rapid development and deployment.

Technologies now exist that allow for the rapid development and deployment of reusable analysis code for use in computerized data modeling and analysis. One such existing technique requires a centralized, continuously updated environment that captures the pre-processing steps used in analyzing big data. Such capture helps accomplish complex transformations and calculations that are continually fresh and accessible to those investigating business opportunities. However, such existing techniques provide reuse analysis of only the preprocessing step, but not of intermediate steps of the preprocessing step. Therefore, such existing techniques do not allow reuse of pre-stored code samples at the unit level, i.e., at the intermediate step level, but only at a higher level, i.e., at the preprocessing step. do. Reuse of pre-stored code samples at the pre-processing step level is disadvantageous and such re-use is therefore only possible when the entire pre-processing step matches the pre-processing step of pre-stored code samples. It is. As an example, consider that the pre-stored code sample for preprocessing step "XYZ" includes intermediate steps 1, 2, and 3. Suppose a data engineer wants to reuse code samples for preprocessing step "ABC", which includes intermediate steps 1, 3, and 6. In such a case, the reuse analysis performed with existing technology is at the preprocessing step level and not at the intermediate step level, so intermediate steps 1 and 3 of the two preprocessing steps "ABC" and "XYZ" are common. However, data engineers may not be able to reuse pre-stored code samples for pre-processing steps "XYZ". Furthermore, in such existing techniques, the code samples pre-stored for reuse are user-defined. Accordingly, pre-stored code samples provided for reuse are based on user preferences for defining reuse code samples, thereby reducing the number of code samples available for reuse.

Other existing techniques also rarely disclose macroservice algorithm-based reuse. Such existing techniques convert the source code of an analysis program that includes a set of operational units into a graphical representation and generate code for a macro service based on a determined subset of the graphical representation. However, these existing techniques allow reuse at the algorithm level, but not at the preprocessing step level or at the intermediate step level. Furthermore, these existing techniques allow the generation of code samples for reuse based on the final algorithmic analysis steps used, but not on the preliminary analysis steps encountered while developing the algorithm. does not enable generation of code samples for reuse. Therefore, the code samples available for reuse, and the scope of such code samples, are substantially less, which risks not being able to meet the software reuse requirements of users in rapid software development environments. It's affecting me.

Therefore, there is a need to enable reuse of code samples at intermediate step levels and to improve the targeting of code samples available for reuse. It is for the purpose of improving the understanding of the general background of the invention only and is not to be construed as an admission or any form of suggestion that this information constitutes prior art already known to those skilled in the art.

Disclosed herein is a method of performing reuse analysis for model lifecycle management. The method includes receiving input requirements from a user via a user device by a reuse analysis system. The input requirements facilitate determining a corresponding reuse package from multiple reuse packages. A plurality of reuse packages are generated for each of the one or more intermediate steps associated with each of the one or more data processing actions. Upon receiving the input requirements, the method includes determining a reuse analysis configuration corresponding to a type of reuse package that is appropriate for the input requirements. Thereafter, the method performs a semantic similarity analysis of the input requirements using one or more pre-stored metadata associated with the plurality of reuse packages based on the reuse analysis configuration. including determining one or more candidate reuse packages from the reuse packages. Finally, the method includes providing one or more candidate reuse packages as a suggestion to the user.

Additionally, the present disclosure discloses a reuse analysis system for performing reuse analysis for model lifecycle management. The reuse analysis system includes a processor and memory communicatively coupled to the processor. The memory stores processor instructions that, when executed, cause the processor to receive input requirements from a user via a user device. The input requirements facilitate determining a corresponding reuse package from multiple reuse packages. A plurality of reuse packages are generated for each of the one or more intermediate steps associated with each of the one or more data processing actions. Upon receiving the input requirements, the processor determines a reuse analysis configuration that corresponds to a type of reuse package that is appropriate to the input requirements. The processor then performs a semantic similarity analysis of the input requirements using one or more pre-stored metadata associated with the plurality of reuse packages based on the reuse analysis configuration. One or more candidate reuse packages are determined from the reuse packages. Finally, the processor provides one or more candidate reuse packages as a suggestion to the user.

The above summary of the invention is for illustrative purposes only and is not intended to be limiting in any way. In addition to the exemplary aspects, embodiments, and features described above, additional aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.

The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate example embodiments and, together with the description, serve to explain the disclosed principles. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the figures to refer to similar features and components. Some embodiments of systems and/or methods according to embodiments of the present subject matter will now be described, by way of example only, and with reference to the accompanying figures.

FIG. 3 illustrates an example architecture for performing reuse analysis for model lifecycle management in accordance with some embodiments of the present disclosure. FIG. 2 illustrates a detailed block diagram of a package generation system for generating multiple reuse packages needed to perform reuse analysis, in accordance with some embodiments of the present disclosure. 3A and 3B are diagrams illustrating an example predetermined navigation schema and an example directed data dependency state diagram, respectively, in accordance with some embodiments of the present disclosure; FIG. 3A and 3B are diagrams illustrating an example predetermined navigation schema and an example directed data dependency state diagram, respectively, in accordance with some embodiments of the present disclosure; FIG. FIG. 3 is a diagram illustrating exemplary first metadata in accordance with some embodiments of the present disclosure. FIG. 4 illustrates an example action execution sequence constructed based on example second metadata in accordance with some embodiments of the present disclosure. FIG. 3 is a diagram illustrating an example predetermined relationship schema according to some embodiments of the present disclosure. FIG. 7 is a diagram illustrating exemplary third metadata according to some embodiments of the present disclosure. 1 illustrates a detailed block diagram of a reuse analysis system for performing reuse analysis for model lifecycle management in accordance with some embodiments of the present disclosure. FIG. FIG. 3 illustrates a flowchart illustrating a method for generating multiple reuse packages needed to perform reuse analysis according to some embodiments of the present disclosure. FIG. 3 depicts a flowchart illustrating a method of performing reuse analysis for model lifecycle management in accordance with some embodiments of the present disclosure. 1 illustrates a block diagram of an example computer system for implementing embodiments consistent with this disclosure. FIG.

It should be understood by those skilled in the art that any block diagrams herein represent conceptual illustrations of example systems implementing the principles of the present subject matter. Similarly, any flowchart, flowchart, state diagram, pseudocode, etc. substantially represented on a computer-readable medium and executed by a computer or processor, whether or not such computer or processor is explicitly depicted. It can be understood to represent a variety of processes for obtaining.

The word "exemplary" is used herein to mean "serving as an example, example, or illustration." Any embodiment or implementation of the subject matter described herein as "exemplary" is not necessarily to be considered preferred or advantageous over other embodiments.

While the disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will be described in detail below. However, it is not intended to limit this disclosure to the particular form disclosed, but on the contrary, this disclosure includes all modifications, equivalents, and alternatives falling within the scope of this disclosure. I hope you understand that.

The term "comprises," "comprising," or any other variation thereof is intended to cover inclusiveness, thereby referring to a setup that includes a list of components or steps; A device or method may include not only those components or steps, but also other components or steps not explicitly shown or inherent to such setup or device or method. In other words, one or more elements of a system or apparatus that are preceded by "comprises...a" may be used with other elements of the system or method without further restriction. does not exclude the presence of elements or additional elements.

Disclosed herein are methods and apparatus for performing reuse analysis for model lifecycle management. The method includes receiving input requirements from a user device associated with a user by a reuse analysis system. In some embodiments, the input requirements are business requirements (BR), analysis requirements (AR), and data requirements (DR) that may facilitate determining a corresponding reuse package from multiple reuse packages. It's fine. In some embodiments, multiple reuse packages may be generated for each of the one or more intermediate steps associated with each of the one or more data processing actions. In some embodiments, multiple reuse packages may be generated by a package generation system associated with a reuse analysis system.

In some embodiments, the package generation system may generate multiple reuse packages by first providing a data repository containing raw historical data in an easy-to-use format. In some embodiments, providing raw historical data in an easy-to-use format may include generating first metadata that indicates parameters, constraints, and relationships of the historical data. In some embodiments, the first metadata may be a retention type. Then, when the package generation system receives a data query from the user, the package generation system uses an easy-to-use Can parse historical data in formats. In the meantime, the packet generation system provides a snapshot of the state of historical data in the data repository, the data queries, the process data, and the relationships between the process data and the corresponding data processing actions in which the process data will be used. Can generate shots. In some embodiments, the package generation system may receive a request from a user to perform one or more data processing actions on process data, and during the data processing action, the package generation system may perform one or more data processing actions on process data. An action execution sequence may be constructed that incorporates each of one or more intermediate steps involved in performing a processing action. The package generation system may generate an action execution sequence based on second metadata generated for each of the one or more intermediate steps and a flow defined in a predetermined relationship schema. In some embodiments, the second metadata includes, in a continuous flow, the steps, actions, referenced modules, dependencies, relationships, etc. involved in performing each of the one or more intermediate steps. may indicate an encoded record. In some embodiments, the second metadata may be of a descriptive type. Thereafter, the package generation system may generate third metadata for each of the one or more data processing actions based primarily on the first metadata, the second metadata, and the action execution sequence. In some embodiments, the third metadata may be a description type. In some embodiments, the package generation system may generate multiple reuse packages based on the third metadata. In some embodiments, the third metadata may indicate data for designing/building the reuse package. In some embodiments, each reuse package may include multiple reuse codes corresponding to each of the one or more intermediate steps involved in performing each data processing action. The first metadata, the second metadata, and the third metadata generated by the package generation system are referred to in subsequent sections as one or more pre-stored metadata in the memory of the package generation system. can be stored in In some embodiments, the one or more pre-stored metadata is used to perform a reuse evaluation, or in other words, to determine one or more candidate reuse packages that are appropriate for the input requirements. can be taken out.

Accordingly, upon receiving input requirements, the reuse analysis system uses one or more pre-stored metadata associated with the plurality of reuse packages based on a reuse analysis configuration appropriate to the input requirements. One or more candidate reuse packages may be determined from the plurality of reuse packages by performing a semantic similarity analysis of the input requirements. In some embodiments, one or more candidate reuse packages thus determined may be provided as a suggestion to a user.

The present disclosure records each intermediate step involved in performing a data processing action and generates a reuse package corresponding to each intermediate step. This enables reuse across data processing, analysis, and application layers, which also helps speed up the process of deploying and creating applications for multiple points of use. Because a reuse package is generated for each intermediate step, the present disclosure differs from existing techniques that mandate similarity of user input requirements to the complete main step to enable reuse. Enables reuse from intermediate step levels themselves. Therefore, the present disclosure provides a higher reuse probability since the reuse package is not limited to only the main steps involved in performing data processing actions. The present disclosure also checks the information labels present in the third metadata based on the configuration type (also referred to in this disclosure as reuse analysis configuration) of the input requirements received from the user, and the information labels present in the third metadata are is useful for determining one or more candidate reuse packages among reuse packages. Such a search process narrows the search to only reuse packages belonging to the configuration type of the input requirements, thereby achieving an easy, quick, and accurate determination of one or more candidate reuse packages for use. The present disclosure also takes and persists snapshots of data at each step, intermediate step, action, etc. based on which reuse package is generated. Such data persistence and snapshots may also include data related to preliminary steps and actions. Preliminary steps and actions could be defined as trial steps and actions performed by the user before arriving at the correct steps and actions. Therefore, reuse packages are also generated based on preliminary steps and actions, which likewise serves to increase the number and scope of reuse packages. Further, in this disclosure, reuse packages are automatically generated by the package generation system for each intermediate step involved while performing data processing actions. Therefore, manual intervention to define/select the data used to generate the reuse package is completely eliminated, thereby expanding the scope of the reuse package for use.

In general, the present disclosure eliminates manual intervention, expands the scope of reuse packages, ensures higher reuse probability by introducing reuse packages from the intermediate step level itself, and determines the configuration type of input requirements. enables quick and accurate retrieval of candidate reuse packages. Accordingly, the present disclosure helps to significantly reduce the time and resources involved in developing software code, as well as the costs involved in managing newly developed software.

In the following detailed description of embodiments of the present disclosure, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration certain embodiments in which the present disclosure may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure, and it is understood that other embodiments may be utilized and changes may be made without departing from the scope of the disclosure. I want you to understand. Therefore, the following description should not be construed in a limiting sense.

FIG. 1 illustrates an example architecture for performing reuse analysis for model lifecycle management according to some embodiments of the present disclosure.

The architecture 100 includes data sources 101 ₁ through data sources 101 _n (collectively referred to as one or more data sources 101), a data repository 103, a package generation system (PGS) 105, a package repository 107, and a reuse analysis system ( RAS) 109, user device 111, and user 113. In some embodiments, one or more data sources 101 may be a repository associated with a business entity, such as, for example, a medical repository, a transportation repository, a financial repository, a restaurant repository, a telecommunications repository, a blockchain records repository, etc. . In some embodiments, the PGS 105 receives raw data from each of the one or more data sources 101 associated with the PGS 105 and stores the raw data as historical data in a data repository 103 associated with the PGS 105. obtain. In some embodiments, raw data may be data in its original form, such as, for example, in an unclassified format, a cluttered format, a different format, etc. Such raw data requires transformation into a specific, easy-to-use format, one in which the data and relationships, dependencies within the data, location of the data, etc. are well defined, connected, and categorized. and this makes the data usable. In some embodiments, PGS 105 may communicate with one or more data sources 101 via a communication network (not shown in FIG. 1). The communication network may be one of a wired communication network, a wireless communication network, or a combination of both wired and wireless communication networks. In some embodiments, PGS 105 may access historical data from data repository 103 to generate multiple reuse packages. In some embodiments, multiple reuse packages may be used to perform reuse analysis.

PGS 105 may include a PGS processor 115, a PGS input/output (I/O) interface 117, and a PGS memory 119, as shown in FIG. PGS I/O interface 117 may provide data-as-a-service obtained by converting historical data from a raw format to an easy-to-use format. In some embodiments, PGS processor 115 may generate the first metadata needed to generate multiple reuse packages while converting historical data. In some embodiments, the first metadata may be a retention type. In some embodiments, the first metadata may include, but is not limited to, historical data parameters, constraints, and relationships. Providing data as a service, PGS processor 115 may receive data queries from user 113 via user device 111. In some embodiments, data queries may relate to process data needed to perform one or more data processing actions by PGS 105. As an example, one or more data processing actions may include, but are not limited to, sampling, filtration, aggregation, and the like. PGS processor 115 may parse historical data in a convenient format to fetch process data needed to perform one or more data processing actions based on a data query.

Also, in parallel, the PGS processor 115 executes one or more data processing actions and constructs an action execution sequence for each of the one or more intermediate steps associated with each of the corresponding data processing actions. One or more state snapshots may be created. In some embodiments, the one or more state snapshots capture the current state of the entity, e.g., the one or more state snapshots capture the state of historical data in the data repository 103, the data queries, the data queries, etc. and the relationship between the process data and the corresponding data processing action in which the process data would be used. In some embodiments, PGS processor 115 may generate second metadata for each of the one or more intermediate steps while building the action execution sequence. In some embodiments, the second metadata may be of a descriptive type. In some embodiments, the second metadata includes, in a continuous flow, the steps, actions, referenced modules, dependencies, relationships, etc. involved in performing each of the one or more intermediate steps. May indicate encoded records. Thereafter, PGS processor 115 may generate third metadata for each of the one or more data processing actions based primarily on the first metadata, the second metadata, and the action execution sequence. In some embodiments, the third metadata may be a description type. In some embodiments, the third metadata includes the one or more intermediate steps of the corresponding data processing action, the second metadata generated for each of the one or more intermediate steps, the second a reference to first metadata corresponding to the metadata, an action execution sequence for each of the one or more intermediate steps, an identifier (ID) corresponding to each action execution sequence, each of the one or more intermediate steps and corresponding data. It may include, but is not limited to, predetermined data queries associated with the processing action, the type of reuse package, and a package identifier (ID). In some embodiments, the third metadata may indicate data for designing/building the reuse package. PGS processor 115 may generate a plurality of reuse packages stored in package repository 107 associated with PGS 105 based on the third metadata. In some embodiments, each reuse package may include multiple reuse codes corresponding to each of the one or more intermediate steps involved in performing each data processing action.

Additionally, RAS 109 may be associated with package repository 107, as shown in FIG. In some embodiments, the RAS 109 performs the following based on data queries and requests received from the user(s) 113 via the user device 111 associated with the user(s) 113: Reuse analysis may be performed to determine and retrieve one or more candidate reuse packages from the plurality of reuse packages. In some embodiments, the user 113 communicating with RAS 109 may be the same user 113 communicating with PGS 105. In some other embodiments, different users may be communicating with RAS 109 and PGS 105. In some embodiments, RAS 109 may subsequently provide one or more candidate reuse packages as suggestions to user 113 for reuse in software development.

In some embodiments, RAS 109 includes a RAS processor 121, a RAS input/output (I/O) interface 123, and a RAS memory 125, as shown in FIG. RAS I/O interface 117 may receive input requirements from user 113, which may facilitate determining a corresponding reuse package from a plurality of reuse packages that is appropriate for user 113. In some embodiments, input requirements may include, but are not limited to, business requirements (BR), analysis requirements (AR), and data requirements (DR). In some embodiments, BR may be a high-level user requirement or objective, and AR requires performing data analysis to provide the user 113 with the required results. The DR may be a requirement below a specific BR and AR that requires fetching or preparing data to perform various data processing activities. . In some embodiments, multiple reuse packages may be generated for each of the one or more intermediate steps associated with each of the one or more data processing actions. Upon receiving the input requirements, RAS processor 121 may determine a reuse analysis configuration that corresponds to a type of reuse package suitable for the input requirements. In some embodiments, RAS processor 121 may determine the reuse analysis configuration by mapping input requirements to types of reuse packages using one or more classification techniques.

Additionally, RAS processor 121 performs semantic similarity analysis of the input requirements using one or more pre-stored metadata associated with the plurality of reuse packages based on the reuse analysis configuration. One or more candidate reuse packages may be determined from the plurality of reuse packages. The one or more pre-stored metadata may refer to at least one of first metadata, second metadata, and third metadata, described in subsequent paragraphs of this disclosure. be. In some embodiments, the first metadata, the second metadata, and the third metadata are generated when generating the multiple reuse packages and are based on input requirements received from the user 113. may be stored in at least one of PBS memory 119 and package repository 107 before performing reuse analysis. In some embodiments, the one or more candidate reuse packages are reuse packages whose one or more pre-stored metadata semantically match the input requirements. In some embodiments, each of the one or more candidate reuse packages is assigned a score indicating the degree of semantic match between the input requirements and the one or more pre-stored metadata. It's fine. In some embodiments, the score may assist user 113 in selecting appropriate candidate reuse packages for use. RAS processor 121 may then provide one or more candidate reuse packages as suggestions to user 113 to enable software reuse in software development operations.

FIG. 2A is a detailed block diagram of a package generation system (PGS) for reclaiming multiple reuse packages in accordance with some embodiments of the present disclosure.

In some implementations, PGS 105 may include data 203 and modules 205. As an example, data 203 of PGS 105 is stored in memory 119 configured within PGS 105, as shown in FIG. 2A. In one embodiment, the data 203 of the PGS 105 includes first metadata 207, snapshot data 209, second metadata 211, action sequence data 213, third metadata 215, reuse package data 217, and others. data 219 may be included. 2A, module 205 of PGS 105 is described in detail herein.

In some embodiments, data 203 of PGS 105 may be stored in memory 119 of PGS 105 in the form of various data structures. Additionally, data 203 in PGS 105 may be organized using a data model, such as a relational data model or a hierarchical data model. In some embodiments, other data 219 may store data including temporary data and primary files generated by modules 205 of PGS 105 to perform various functions of PGS 105.

In some embodiments, data 203 of PGS 105 stored in PGS memory 119 may be processed by module 205 of PGS 105. Module 205 may be stored within memory 119 of PGS 105. Also, in one example, module 205 communicatively coupled to PGS processor 115 configured within PGS 105 may reside outside of PGS memory 119 and be implemented as hardware, as shown in FIG. 2A. As used herein, the term module refers to an application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group) that executes one or more software or firmware programs, and memory; Refers to combinational logic circuits and/or other suitable components that provide the described functionality.

In one embodiment, the modules 205 of PGS 105 may include, for example, a transformation module 221, a metadata generation module 223, a snapshot creation module 225, a sequence construction module 227, a package generation module 229, and other modules 231. Other modules 231 may be used to perform a variety of diverse functions of PGS 105. It is to be understood that such above-described modules 205 of PGS 105 may be represented as a single module or a combination of different modules.

In some implementations, conversion module 221 may convert historical data from a raw format to an easy-to-use format to provide data-as-a-service. In some embodiments, historical data may be stored in a data repository 103 associated with PGS 105. As an example, historical data may include data received from one or more data sources 101, such as a medical repository, a transportation repository, a financial repository, a restaurant repository, a telecommunications repository, a blockchain records repository, etc. In some embodiments, to provide data as a service, transformation module 221 may first parse historical data to learn parameters, constraints, and relationships of the historical data. The transformation module 221 may then create a directed data dependency state diagram based on the learned parameters, constraints, and relationships of the historical data and the predetermined data navigation schema. In some embodiments, a predetermined navigation schema provides a common coded format for connecting parameters, constraints, and relationships using data navigation commands, data endpoints, functions, module names, etc. The layout may be as follows. In some embodiments, a directed data dependency state diagram may be a modular structure representing learned parameters, constraints, and relationships based on a layout provided by a predetermined navigation schema. In some embodiments, a directed data dependency state diagram may include, but is not limited to, data identifiers, historical data, data navigation commands, and data endpoints, where each data endpoint is , associated with an application programming interface (API). An example given navigation schema and an example directed data dependency state diagram may be as shown in FIGS. 2B and 2C, respectively. As shown in Figure 2B, a given navigation schema includes the schema title, modules, parameters, resources referenced in the schema such as constraints, types of data representation, links representing data connections and actions, relationships, hypertext references, etc. (href), methods applied to the data, etc. In some embodiments, relationships may be indicated using data navigation commands such as "dive deep," "finish," "penetrate," etc. In some embodiments, "Dig Deep" may be a data navigation command that allows you to move one level deeper while searching for data, and "Finish" allows you to move up one level while searching for data. It may be a data navigation command that allows you to navigate, and "penetrate" may be a navigation command that allows you to penetrate to a higher level while searching for data. In some other embodiments, relationships may indicate how data within a module should be processed. As an example, a relationship designated as "summarize" may mean that data within the specified module needs to be summarized. Similarly, a relationship indicated as "related" may mean that the data within the specified module must be related to each other.

Additionally, the method "get" shown in Figure 2B is a command that applies a procedure for reading and retrieving data from a particular module shown using a data endpoint in parallel with "href://". It's fine.

As shown in Figure 2C, the directed data dependency state graph includes the module's identifier, the data associated with the module, and the data navigation commands for searching the directed data dependency state graph and accessing each module. may contain the data endpoints needed for the In Figure 2C, the arrow mark indicates the direction of data navigation, for example, selecting the data navigation command "Deep Deep" in the first module indicated by "Id1" is followed by one level down, i.e. by "Id2". This may enable navigation to the second module shown, which is illustrated in FIG. 2C using an arrow from "Dig Deep" in the first module to "Id2" in the second module. Similarly, selection of the data navigation command "Exit" in the second module indicated by "Id2" may enable navigation up one level, i.e. to the first module indicated by "Id1"; This is illustrated in Figure 2C using the arrow from "Finish" in the second module to "Id1" in the first module. Similarly, the remainder of the arrows in FIG. 2C indicate data navigation directions.

Additionally, metadata generation module 223 may generate first metadata 207 based on a directed data dependency state diagram and a predetermined data navigation schema to provide data as a service. In other words, according to the learned parameters shown in the directed data dependency state diagram, the historical data represented in the layout provided by the predetermined navigation schema is the first metadata 207 generated by the metadata generation module 223. It may be. FIG. 2D shows example first metadata, including, for example, a "relationship schema for modules," a type of data representation as an "array," and links representing data connections and actions. One of the example links shown in FIG. 2D shows a relationship as "related", a data endpoint as "/model", and a method as "get", which is related to the data endpoint "/model". This means that the data of the specified module stored in "model" needs to be corrected and retrieved. Similarly, another exemplary link shown in FIG. 2D shows a relationship as 'summary', a data point as '/aggregation', and a 'method' as 'get', which connects the data endpoint ' /aggregation" means that the data of the specified module stored in "/aggregation" needs to be summarized and retrieved.

Generally, raw data is data that has been accumulated in a storage area from one or more data sources. As explained above, by identifying parameters, relationships, dependencies, etc., the metadata generation module 223 and associated transformation module 221 convert the raw data into a consumable format, i.e., a meaningful and useful format. data and thus provide data as a service.

In some embodiments, snapshot creation module 225 may create one or more state snapshots while performing one or more data processing actions. As an example, one or more data processing actions may be sampling, filtration, aggregation, etc. In some embodiments, providing data as a service allows PGS 105 to receive data queries from users to retrieve process data necessary to perform one or more data processing actions. In some embodiments, the data query may be in the original data format. Snapshot creation module 225 may convert the received data query from its original data format to a searchable index format that represents data endpoints. Once converted to a searchable index format, the snapshot creation module 225 may mine the historical data (provided as data-as-a-service) to identify process data that corresponds to data queries from each data endpoint. ) may provide the process data along with a storage identifier and location of the process data within the data repository 103; In some embodiments, process data may be a subset of historical data needed to perform one or more data processing actions.

For example, consider a data entry in the original format as shown below.
．． csv, data data 1, range>50<100

The data query converted from its original format to a searchable index format may be as shown below.
href://module1, parameter, constraint

In the above searchable index format, the parameter is "data 1" and the constraint is that the parameter must be in a range greater than 50 but less than 100. Therefore, the requirement is considered a data requirement (DR) and the process data provided as output to the user 113 is from the storage location referenced by the data endpoint href://module1 within the constraints indicated by the data range. It will be data. As an example, a query to retrieve the output process data as described above, provided by PGS 105 to data repository 103 in this example, would be as follows.
DR: Data 1 to return range > 50 < 100
Storage Id: <324856>
Location: /data/module/? m1

Additionally, upon providing the process data corresponding to the data query, the snapshot creation module 225 determines the state of the historical data in the data repository 103, the data query, the process data, and the corresponding data for which the process data is used. One or more state snapshots may be created to store relationships with actions. In some embodiments, one or more snapshots are taken within a data repository after or while retrieving process data needed to perform one or more data processing actions based on a data query. may be required to understand the state of historical data. In some embodiments, one or more state snapshots may be stored as snapshot data 209.

In some embodiments, sequence construction module 227 may construct an action performance sequence for each of the one or more intermediate steps associated with each corresponding data processing action. To construct such an action execution sequence, sequence construction module 227 may first receive a request from user 113 to perform one or more data processing actions on process data. As an example, one or more data processing actions may involve sampling, filtration, aggregation, loading, etc. In some embodiments, while performing one or more data processing actions, sequence construction module 227 identifies a step identifier for each of the one or more intermediate steps involved in each of the one or more data processing actions. can be created. A step identifier may be a numeric, alphabetic, or alphanumeric code given to each intermediate step of one or more processing actions in the order in which they are performed. Metadata generation module 223 may then generate second metadata 211 for each of the one or more intermediate steps based on the corresponding data processing action, step identifier, and process data. In some embodiments, the second metadata 211 includes, in a continuous flow, the steps, actions, referenced modules, dependencies, relationships, etc. involved in performing each of the one or more intermediate steps. may indicate an encoded record. The second metadata 211 generated in this manner may be stored in the memory 119. Upon generating the second metadata 211, the sequence construction module 227 may construct an action execution sequence based on the second metadata 211 and the flow defined in the predetermined relationship schema. The action execution sequence thus constructed for each request to perform one or more data processing actions received from user 113 may be stored as action sequence data 213. FIG. 2E shows an example action execution sequence constructed based on the example second metadata 211 as shown in FIG. 2E. Further, FIG. 2F depicts an example predetermined relationship schema that illustrates the flow between example data processing actions such as data sampling, data aggregation, data filtration, model building, loader, and the like. The arrow marks between each of these example data processing actions in FIG. 2F indicate the relationship between the different data processing actions. As an example, the arrow "Associate" between data sampling and model indicates that the sampled data is correlated with a given model, and the arrow "Summary" between model and data aggregation indicates that the model may indicate that it is fetching summarized aggregation data. Using such a predetermined relationship schema, sequence construction module 227 may generate an action execution sequence as shown in FIG. 2E.

In some embodiments, metadata generation module 223 may generate third metadata 215 for each of the one or more data processing actions. The third metadata 215 includes one or more intermediate steps of the corresponding data processing action, second metadata 211 generated for the one or more intermediate steps, and third metadata 215 that corresponds to the second metadata 211. 1, an action execution sequence for each of the one or more intermediate steps, an identifier (ID) corresponding to each action execution sequence, an identifier (ID) associated with each of the one or more intermediate steps and the corresponding data processing action. may include, but are not limited to, predetermined data queries, types of reuse packages, and package identifiers (IDs). In some embodiments, third metadata 215 may be generated using all the information described above as content of third metadata 215. In some embodiments, third metadata 215 may indicate data that designs/builds example reuse package "P-01." As an example, the exemplary third metadata of FIG. 2G indicates that "P-01" is associated with the business requirement (BR) "Customer Classification Based on Payment Analysis." Within this reuse package "P-01" there are intermediate steps, and the exemplary third metadata indicates step Ids such as "S1", "S3", "S4", etc. of each intermediate step. . In some embodiments, a step Id was assigned to each intermediate step during the process of generating the action execution sequence. In some embodiments, each intermediate step may be associated with a particular type of requirement. As an example, in Figure 2G, one of the intermediate steps "Sampling" includes data requirements (DR )is connected with. Similarly, the intermediate steps ``Model'' include ``Obtain customer segments predicted to increase their payments by more than Rs. related to analysis requirements (ARs) such as ``Obtain customer segments predicted to decrease

In some embodiments, using such third metadata 215 generated by metadata generation module 223, package generation module 229 may generate multiple reuse packages. In some embodiments, multiple reuse packages may be generated for each intermediate step involved in the data processing step, and such packages may also be associated with different requirements, such as business requirements, miracle requirements, data requirements, etc. . A plurality of reusable packages generated in this manner can be stored as reusable package data 217. In some embodiments, multiple reuse packages may then be stored in a package repository 107 associated with PGS 105.

FIG. 2H shows a detailed block diagram of a reuse analysis system (RAS) for performing reuse analysis for model lifecycle management in accordance with some embodiments of the present disclosure.

In some implementations, RAS 109 may include data 243 and modules 245. As an example, data 243 of RAS 109 is stored in RAS memory 125 configured in RAS 109 as shown in FIG. 2H. In one embodiment, data 243 in RAS 109 may include requirements data 247, configuration data 249, and candidate reuse package data 251, and other data 253. 2A, module 245 of RAS 109 is described in detail herein.

In some embodiments, RAS 109 data 243 may be stored in RAS memory 125 of RAS 109 in the form of various data structures. Additionally, data 243 in RAS 109 can be organized using a data model, such as a relational data model or a hierarchical data model. In some embodiments, other data 253 may store data including temporary data and temporary files generated by modules 245 of RAS 109 to perform various functions of RAS 109.

In some embodiments, RAS 109 data 243 stored in RAS memory 125 may be processed by RAS 109 module 205. Module 245 may be stored within RAS memory 125 of RAS 109. In one example, module 245 communicatively coupled to PGS processor 115 configured with RAS 109 resides outside of RAS memory 125 and may be implemented as hardware, as shown in FIG. 2H. As used herein, the term module refers to an application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group) that executes one or more software or firmware programs, and memory; Refers to combinational logic circuits and/or other suitable components that provide the described functionality.

In one embodiment, modules 245 of RAS 109 may include, for example, a receiving module 255, a configuration determination module 257, a reuse package determination module 259, and other modules 261. Other modules 261 may be used to perform a variety of diverse functions of RAS 109. It should be understood that such above-described modules 245 of RAS 109 may be represented as a single module or a combination of different modules.

In some embodiments, receiving module 255 may receive input requirements from user device 111 of user 113. Input requirements received in this manner from user 113 may be stored as requirements data 247. The input requirements may facilitate determining a corresponding reuse package from a plurality of reuse packages stored in a package repository 107 associated with RAS 109 and package generation system (PGS) 105. In some embodiments, the PGS 105 performs a process for each of the one or more intermediate steps associated with each of the one or more data processing actions, as described below in the description of FIG. 2H of this disclosure. Multiple reuse packages can be generated. In some embodiments, input requirements may relate to at least one of business requirements (BR), analysis requirements (AR), and data requirements (DR). In some embodiments, the BR may be a high-level requirement or objective of the user, and the AR may be a specific requirement or objective that requires data analysis to be performed to provide the user 113 with the required results. The DR may be a requirement below a particular BR and AR that requires fetching or preparing data to perform various data processing activities. As an example, a BR may be "Customer Classification Based on Payment Analysis" and the AR associated with such BR may be "Customer Classification Based on Payment Analysis" and the AR associated with such BR may be "Customer Classification Based on Payment Amount Analysis" and the AR associated with such BR may be "Customer Classification Based on Payment Amount Analysis" and the AR associated with such BR may be "Customer Classification Based on Payment Amount Analysis" and the AR associated with such BR may be "Customer Classification Based on Payment Amount Analysis" and the AR associated with such BR may be "Customer Classification Based on Payment Amount Analysis" and the AR associated with such BR may be "Customer Classification Based on Payment Amount Analysis"; The DR for such BRs and ARs may be "Get the last three years of customer payment data from the customer table."

In some embodiments, configuration determination module 257 may determine a reuse analysis reconfiguration that corresponds to a type of reuse package that is appropriate to the input requirements. In some embodiments, determining the reuse analysis configuration may include determining whether the input requirement belongs to one of BR, AR, or DR. In some embodiments, configuration determination module 257 may determine reuse analysis configurations by mapping input requirements to types of reuse packages using one or more classification techniques. By way of example and not limitation, the one or more classification techniques may include latent semantic analysis techniques, topic analysis techniques, and the like. In some embodiments, the types of reuse packages include reuse packages associated with AR, reuse packages associated with DR, reuse packages associated with BR, reuse packages associated with a combination of AR, BR, and DR. It may be a usage package, etc. The reuse analysis configuration determined in this manner may be stored as configuration data 249.

Further, the reuse package determination module 259 performs a semantic similarity analysis of the input requirements using one or more pre-stored metadata associated with the plurality of reuse packages based on the reuse analysis configuration. One or more candidate reuse packages may be determined from the plurality of reuse packages. In some embodiments, the one or more candidate reuse packages may be reuse packages whose one or more pre-stored metadata semantically match the input requirements. One or more candidate reuse packages determined in this manner may be stored as candidate reuse package data 251. The one or more pre-stored metadata may be at least one of first metadata, second metadata, and third metadata. In some embodiments, reuse package determination module 259 uses one or more natural language processing (NLP) techniques to determine semantic similarity of input requirements using one or more pre-stored metadata. analysis can be determined. As an example, the one or more NLP techniques may include word frequency-inverse document frequency (TF-IDF) techniques, latent semantic analysis (LSA) techniques, and other similar techniques in the field of NLP. However, it is not limited to these. Additionally, in some embodiments, the reuse package determination module 259 provides one or more candidate reuse packages with semantic matches between the input requirements and the one or more pre-stored metadata. A grade-based score may be assigned. Such scores may assist user 113 in selecting appropriate candidate reuse packages for use. As an example, the user 113 may select an appropriate candidate reuse package based on the highest score, based on the score having a semantic match that exceeds a preset threshold, etc.

As an example, consider the requirements as shown in Table 1 below.

Using one or more classification techniques, RAS 109 may determine a reuse analysis configuration of the input requirements as shown in Table 2 below.

Since the reuse analysis configuration of the input requirement is determined to be a BR, the RAS 109 analyzes the meaning of the input requirement using pre-stored metadata of one or more of the plurality of reuse packages associated with type BR. Similarity analysis can be performed.

As an example, consider that the multiple reuse packages associated with type BR are as shown in Table 3 below.

Upon performing the semantic similarity analysis, RAS 109 may determine one or more candidate reuse packages and corresponding scores as shown in Table 4 below.

According to Table 4 above, input requirements with requirement IDs 1, 2, and 3 are pre-stored metadata of reuse packages P1, P3, and Pn with scores of 99%, 90%, and 78%, respectively. Each semantically matches the data. Therefore, P1 is a candidate reuse package for the input requirement with requirement ID1, P2 is a candidate reuse package for the input requirement with requirement ID2, and Pn is a candidate reuse package for the input requirement with requirement ID3. It's a package.

Similarly, by way of example, consider the requirements as shown in Table 5 below.

Using one or more classification techniques, RAS 109 may determine a reuse analysis configuration of the input requirements as shown in Table 6 below.

Since the reuse analysis configuration of the input requirement is determined to be an AR, the RAS 109 analyzes the semantic similarity of the input requirement using pre-stored metadata of one or more of the plurality of reuse packages associated with type AR. Analysis can be performed.

As an example, consider that the multiple reuse packages associated with type AR are as shown in Table 7 below.

Upon performing the semantic similarity analysis, RAS 109 may determine one or more candidate reuse packages and corresponding scores as shown in Table 8 below.

According to Table 8 above, the input requirements with requirement IDs 1 and 2 semantically match the pre-stored metadata of reuse packages P1 and P3 with scores of 99% and 90%, respectively. Therefore, P1 is a candidate reuse package for the input requirement of requirement ID1, and P3 is a candidate reuse package for the input requirement of requirement ID2.

Similarly, by way of example, consider the requirements as shown in Table 9 below.

Using one or more classification techniques, RAS 109 may determine a reuse analysis configuration for the input requirements as shown in Table 10 below.

Since the reuse analysis configuration of the input requirement is determined to be a DR, the RAS 109 analyzes the semantic similarity of the input requirement using pre-stored metadata of one or more of the plurality of reuse packages associated with type DR. Analysis can be performed.

As an example, consider that the multiple reuse packages associated with type DR are as shown in Table 11 below.

Upon performing the semantic similarity analysis, RAS 109 may determine one or more candidate reuse packages and corresponding scores as shown in Table 12 below.

According to Table 12 above, the input requirements with requirement IDs 1 and 2 semantically match the pre-stored metadata of reuse packages P1 and P3 with scores of 99% and 90%, respectively. Therefore, P1 is a candidate reuse package for the input requirement of requirement ID1, and P3 is a candidate reuse package for the input requirement of requirement ID2.

FIG. 3A depicts a flowchart illustrating a method of generating multiple reuse packages in accordance with some embodiments of the present disclosure.

As illustrated in FIG. 3A, method 300a includes one or more blocks that illustrate how to generate multiple reuse packages. Method 300a may be described in the general context of computer-executable instructions. Generally, computer-executable instructions may include routines, programs, objects, components, data structures, procedures, modules, and functions that perform functions or implement abstract data types.

The order in which method 300a is described is not intended to be construed as a limitation, and any number of method blocks described may be combined in any order to implement method 300a. Additionally, individual blocks may be deleted from the method without departing from the spirit and scope of the subject matter described herein. Additionally, method 300a may be implemented in any suitable hardware, software, firmware, or combinations thereof.

At block 301, the method includes providing data as a service by a processor 115 of a package generation system (PGS) 103 (also referred to as a PGS processor 115). In some embodiments, PGS processor 115 provides data-as-a-service by converting historical data from a raw format to an easy-to-use format. In some embodiments, PGS processor 115 may receive historical data from one or more data sources 101 and stored in a data repository 103 associated with PGS 105. Initially, PGS processor 115 may parse historical data to learn parameters, constraints, and relationships of the historical data. PGS 115 may then create a directed data dependency state diagram based on learned parameters, constraints, and relationships of historical data and a predetermined data navigation schema. In some embodiments, the directed data dependency state diagram includes data identifiers, historical data, data navigation commands, and data endpoints, and each data endpoint is associated with an application programming interface (API). In some embodiments, the PGS processor 115 may generate the first metadata needed to generate multiple reuse packages while converting the historical data from a raw format to a usable format. . PGS processor 115 may generate first metadata based on a directed data dependency state diagram and a predetermined data navigation schema.

At block 303, the method includes creating one or more state snapshots while performing one or more data processing actions by PGS processor 115. To create one or more state snapshots, PGS processor 115 may receive a data query in its original data format and convert the data query from the original data format to a searchable index format that represents data endpoints. . Additionally, the PGS 115 reviews the historical data to retrieve process data corresponding to the data query from each data endpoint, along with the storage identifier and location of the process data within the data repository 103, and provides the process data to the user 113. can be provided. Upon providing process data, PGS 115 stores the state of historical data in PGS memory 119, data queries, process data, and relationships between the process data and the corresponding data actions in which the process data is used, thereby Create one or more state snapshots. PGS 115 may then construct an action execution sequence for each of the one or more intermediate steps associated with each corresponding data processing action. To build an action execution sequence, PGS processor 115 may first perform one or more data processing actions on process data based on requests received from user 113. PGS 115 creates a step identifier for each of the one or more intermediate steps involved in each of the one or more data processing actions, and creates an action execution sequence based on the corresponding data processing action, step identifier, and process data. The second metadata may be generated for each of the one or more intermediate steps while constructing the second metadata. PGS processor 115 may then construct an action execution sequence based on the second metadata and the flow defined in the predetermined relationship schema and store the constructed action execution sequence for each request.

At block 305, the method includes generating third metadata for each of the one or more data processing actions by the PGS processor 115. In some embodiments, the third metadata includes the one or more intermediate steps of the corresponding data processing action, the second metadata generated for each of the one or more intermediate steps, the second a reference to first metadata corresponding to the metadata, an action execution sequence for each of the one or more intermediate steps, an identifier (ID) corresponding to each action execution sequence, each of the one or more intermediate steps and corresponding data. It may include, but is not limited to, predetermined data queries associated with the processing action, the type of reuse package, and a package identifier (ID).

FIG. 3B depicts a flowchart illustrating a method for performing reuse analysis for model lifecycle management, according to some embodiments of the present disclosure.

As illustrated in FIG. 3B, method 300b includes one or more blocks that illustrate how to perform reuse analysis for model lifecycle management. Method 300b may be described in the general context of computer-executable instructions. Generally, computer-executable instructions may include routines, programs, objects, components, data structures, procedures, modules, and functions that perform functions or implement abstract data types.

The order in which method 300b is described is not intended to be construed as a limitation, and the number of method blocks described may be combined in any order to implement method 300b. Additionally, individual blocks may be deleted from the method without departing from the spirit and scope of the subject matter described herein. Additionally, method 300b may be implemented in any suitable hardware, software, firmware, or combinations thereof.

At block 311, the method includes receiving input requirements from a user 113 by a reuse analysis system (RAS) processor 121 (also referred to as RAS processor 121). As an example, the input requirements may be one of business requirements (BR), analysis requirements (AR), and data requirements (DR). In some embodiments, the input requirements facilitate determining a corresponding reuse package from multiple reuse packages. Multiple reuse packages may be generated for one or more intermediate steps associated with each of the one or more data processing actions.

At block 313, the method includes determining, by RAS processor 121, a reuse analysis configuration corresponding to a type of reuse package that is appropriate to the input requirements. In some embodiments, RAS processor 121 may determine the reuse analysis configuration by mapping input requirements to types of reuse packages using one or more classification techniques.

At block 315, the method performs a semantic similarity analysis of the input requirements using the plurality of reuse packages and the one or more pre-stored metadata associated with the reuse analysis configuration based on the reuse analysis configuration. The steps include determining one or more candidate reuse packages from a plurality of reuse packages. In some embodiments, the one or more candidate reuse packages are configured such that the one or more pre-stored metadata are semantically related to input requirements. is a reusable package that matches. In some embodiments, one or more candidate reuse packages are assigned a score based on the degree of semantic match between the input requirements and the one or more pre-stored metadata.

At block 317, the method includes providing one or more candidate reuse packages as a suggestion to user 113 by RAS processor 121. In some embodiments, RAS processor 121 may provide a score with each of the one or more candidate reuse packages provided as a suggestion to user 113.

FIG. 4 is a block diagram of an exemplary computer system for implementing implementations consistent with this disclosure.

In some embodiments, FIG. 4 depicts a block diagram of an exemplary computer system 400 for implementing embodiments consistent with the present invention. In some embodiments, computer system 400 may be reuse analysis system 109 used to perform reuse analysis for model lifecycle management. In some embodiments, computer system 400 may be a package generation system 105 associated with a reuse analysis system 109 that is used to generate multiple reuse packages. FIG. 4 is shown considering computer system 400 as reuse analysis system 109. In FIG. However, this should not be considered a limitation, as computer system 400 may also be package generation system 105 (not shown in FIG. 4). Computer system 400 may include a central processing unit (“CPU” or “processor”) 402. Processor 402 may include at least one data processor for executing program components to execute user- or system-generated business processes. A user may include a person, a person using a device such as one included in the present invention, or such a device itself. Processor 402 may include specialized processing units such as an integrated system (bus) controller, memory management controller, floating point unit, graphics processing unit, digital signal processing unit, and the like.

Processor 402 may be placed in communication with input devices 411 and output devices 412 via I/O interface 401 . I/O interface 401 may include, without limitation, audio bus, analog bus, digital bus, stereo bus, IEEE-1394 bus, serial bus, universal serial bus (USB), infrared, PS/2, BNC, coaxial, component O, Composite, Digital Visual Interface (DVI), High Definition Multimedia Interface (HDMI(R)), Radio Frequency (RF) Antenna, S-Video, Video Graphics Array (VGA), IEEE802. n/b/g/n/x, Bluetooth, cellular (e.g. Code Division Multiple Access (CDMA), High Speed Packet Access (HSPA+), Global Service for Mobile Communications (GSM), Long Term Evolution (LTE), WiMax, etc. ) and other communication protocols/methods may be utilized.

Using I/O interface 401, computer system 400 may communicate with input devices 411 and output devices 412.

In some embodiments, processor 402 may be placed in communication with communication network 409 via network interface 403. Network interface 403 may communicate with communication network 409. Network interface 403 can include, without limitation, direct connect, Ethernet (e.g., twisted pair 10/100/1000 Base-T), Transmission Control Protocol/Internet Protocol (TCP/IP), Token Ring, IEEE 802.11a/b/g/n/ Connection protocols may be utilized, including x and the like. Using network interface 403 and communication network 409, computer system 400 may communicate with package generation system 105, package repository 107, and user device 111. Package generation system 105 may further be associated with one or more data sources 101 (101 ₁ -101 _n ) and data repositories 103 . In some embodiments, computer system 400 may communicate via IP-based communications and non-Internet or non-IP-based communications, such as Universal Serial Bus (USB), Bluetooth, and the like. Communication network 409 can be implemented as one of different types of networks, such as an intranet or local area network (LAN), a closed area network (CAN), and the like in autonomous vehicles. The communication network 409 can be either a dedicated network or a shared network, which uses e.g. Hypertext Transfer Protocol (HTTP), CAN protocol, Transmission Control Protocol/Internet Protocol (TCP/IP) to communicate with each other. , represents an association of different types of networks using various protocols such as Wireless Application Protocol (WAP). Additionally, communication network 409 may include various network devices including routers, bridges, servers, computing devices, storage devices, and the like. In some embodiments, processor 402 may be arranged in communication with memory 405 (eg, RAM, ROM, etc. not shown in FIG. 4) via storage interface 404. Storage interface 404 may include, without limitation, Serial Advanced Technology Attachment (SATA), Integrated Drive Electronics (IDE), IEEE-1394, Universal Serial Bus (USB), Fiber Channel, Small Computer Peripheral Interface (SCSI), etc. Connection protocols may be utilized to connect to memory 405, including memory drives, removable disk drives, and the like. Memory drives may further include drums, magnetic disk drives, magneto-optical drives, optical drives, RAID, solid state memory devices, solid state drives, and the like.

Memory (405) may store a collection of program or database components including, without limitation, a user interface 406, an operating system 407, a web browser 408, and the like. In some embodiments, computer system 400 may store user/application data such as data, variables, records, etc. as described in the present invention. Such a database may be implemented as a fault-tolerant, relational, scalable, and secure database, such as Oracle or Sybase.

Operating system 407 may facilitate resource management and operation of computer system 400. Examples of operating systems 407 include, without limitation, APPLE® MACINTOSH® OS (BSD), FREEBSD(R), NETBSD(R), OPENBSD, etc.), LINUX(R) DISTRIBUTIONS (e.g., RED HAT(R), UBUNTU(R), KUBUNTU(R), etc.), IBM (registered trademark) OS/2 (registered trademark), MICROSOFT (registered trademark) WINDOWS (registered trademark) (XP (registered trademark), VISTA (registered trademark)/7/8, 10, etc.), APPLE (registered trademark) IOS (registered trademark), GOOGLE (trademark), ANDROID (registered trademark), BLACKBERRY (registered trademark) OS, etc. User interface 406 may facilitate displaying, executing, interacting with, manipulating, or operating program components through textual or graphical features. For example, user interface 406 may provide computer interaction interface elements on a display system operably connected to computer system 400, such as cursors, icons, check boxes, menus, scrollers, windows, widgets, and the like. Apple® Macintosh® operating systems Aqua®, IBM® OS/2®, Microsoft® Windows® (e.g., Aero, Metro, etc.) , graphical user interfaces (GUI), including web interface libraries (e.g., ActiveX®, Java®, JavaScript®, AJAX, HTML, Adobe® Flash®, etc.), etc. ) can be used.

In some embodiments, computer system 400 may implement program components stored in web browser 408. The web browser 408 is a hyperlink such as MICROSOFT (registered trademark) INTERNET EXPLORER (registered trademark), GOOGLE (trademark) CHROME (trademark), MOZILLA (registered trademark) FIREFOX (registered trademark), APPLE (registered trademark) SAFARI (registered trademark), etc. It may be a text display application. Secure web browsing may be provided using Secure Hypertext Transfer Protocol (HTTPS), Secure Sockets Layer (SSL), Transport Layer Security (TLS), and the like. Web browser 408 may utilize features such as AJAX, DHTML, ADOBE® FLASH®, JAVASCRIPT®, JAVA®, application programming interfaces (APIs), and the like. In some embodiments, computer system 400 may implement program components stored on a mail server. The mail server may be an Internet mail server such as Microsoft Exchange. Mail servers include Active Server Pages (ASP), ACTIVEX®, ANSI® C++/C#, MICROSOFT®, . NET, CGI SCRIPTS, JAVA (registered trademark), JAVASCRIPT (registered trademark), PERL (registered trademark), PHP, PYTHON (registered trademark), WEBOBJECTS (registered trademark), and the like. Mail servers may utilize communication protocols such as Internet Message Access Protocol (IMAP), Messaging Application Programming Interface (MAPI), MICROSOFT Exchange, Post Office Protocol (POP), Simple Mail Transfer Protocol (SMTP), and the like. In some embodiments, computer system 400 may implement program components stored in a mail client. The email client is an email display application such as APPLE (registered trademark) MAIL, MICROSOFT (registered trademark) ENTOURAGE (registered trademark), MICROSOFT (registered trademark) OUTLOOK (registered trademark), MOZILLA (registered trademark) THUNDERBIRD (registered trademark), etc. It's fine.

Additionally, one or more computer-readable storage media may be utilized in implementing embodiments consistent with the present invention. Computer-readable storage media refers to any type of physical memory that can store information or data that can be read by a processor. Accordingly, a computer-readable storage medium includes instructions for execution by one or more processors to cause the processor(s) to perform steps or stages consistent with the embodiments described herein. May store instructions. The term "computer-readable medium" is to be understood to include tangible items and exclude carrier waves and transient signals, ie, non-transitory. Examples include random access memory (RAM), read-only memory (ROM), volatile memory, non-volatile memory, hard drives, compact discs (CDs) ROMs, digital video discs (DVDs), flash drives, disks, and any including other known physical storage media.

Advantages of embodiments of the present disclosure are presented here.

The present disclosure records each intermediate step involved in performing a data processing action and generates a reuse package corresponding to each intermediate step. This enables reuse across data processing, analysis, and application layers, which in turn helps speed up the process of deploying and creating applications for multiple points of use. Because a reuse package is generated for each intermediate step, the present disclosure differs from existing techniques that mandate similarity of user input requirements to the complete main step to enable reuse. Enables reuse from intermediate step levels themselves. Therefore, the present disclosure provides a higher reuse probability since reuse packages are not limited to the main steps involved in performing data processing actions.

Further, in this disclosure, based on the configuration type of the input requirement received from the user, an information label present in the third metadata is checked, which identifies one or more candidate reuses among multiple reuse packages. Help you decide on a package. Such a search process narrows the search to only reuse packages that belong to the configuration type of the input requirements, thereby achieving an easy, quick, and accurate determination of one or more candidate reuse packages for use.

Additionally, the present disclosure takes and persists snapshots of data at each step, intermediate step, action, etc. based on which reuse package is generated. Such data persistence and snapshots may capture data related to preliminary steps and actions as well. Preliminary steps and actions could be defined as trial steps and actions performed by the user before arriving at the correct steps and actions. Therefore, reuse packages are also generated based on preliminary steps and actions, which likewise serves to increase the number and scope of reuse packages.

Further, in this disclosure, reuse packages are automatically generated by the package generation system for each intermediate step involved while performing data processing actions. Therefore, manual intervention to define/select the data used to generate the reuse package is completely eliminated, thereby expanding the scope of the reuse package for use.

In general, the present disclosure eliminates manual intervention, expands the scope of reuse packages, ensures higher reuse probability by introducing reuse packages from the intermediate step level itself, and determines the configuration type of input requirements. enables quick and accurate retrieval of candidate reuse packages. Accordingly, the present disclosure helps to significantly reduce the time and resources involved in developing software code, as well as the costs involved in managing newly developed software.

A description of an embodiment having several components in communication with each other does not imply that all such components are required. On the contrary, various optional components are described to illustrate the wide variety of possible embodiments of the invention. When a single device or product is described herein, it is clear that one or more devices/products (whether or not they cooperate) may be used in place of the single device/product. become. Similarly, when multiple devices or products are described herein (regardless of whether they coordinate or not), a single device/product may be used in place of the multiple devices or products, or It will be apparent that a different number of devices/products may be used in place of the number of devices or programs shown. The functions and/or features of a device may be alternatively implemented by one or more other devices not explicitly described as having such functions/features. Therefore, other embodiments of the invention need not include the device itself.

This specification has described methods and systems for performing reuse analysis for model lifecycle management. The steps shown are presented to explain the example embodiments shown, and it should be anticipated that ongoing technological developments will change the manner in which certain functions are performed. be. These examples are presented herein for purposes of illustration and not limitation. Furthermore, the limits of functional fundamentals are arbitrarily defined herein for convenience of explanation. Alternative limits may also be defined, as long as the specified functions and their relationships are properly performed. Alternatives (including equivalents, extensions, variations, deviations, etc. to those described herein) will be apparent to those skilled in the art based on the teachings contained herein. Such alternatives fall within the scope of the disclosed embodiments. Also, the words "comprising," "having," "containing," and "including" and other similar forms are intended to be equivalent in meaning; The item or items that follow any one of these words are not intended to be a limited enumeration of such item or items, or only the item or items listed. It is open-ended in that it is not intended to be limited. Also, as used in this specification and the appended claims, the singular forms "a," "a," and "the" include plural references unless the context clearly dictates otherwise. Please note that.

Finally, the language used herein has been chosen primarily for readability and educational purposes; the language may not have been chosen to elaborate or limit the subject matter of the invention. . Accordingly, it is intended that the scope of the invention be limited not by this detailed description, but rather by any claims arising on this application. Accordingly, the embodiments of the invention are intended to be illustrative rather than restrictive of the scope of the invention, which is set forth in the claims that follow.

100 Architecture 101 One or more data sources 103 Data repositories 105 Package Generation System (PGS)
107 Package Repository 109 Reuse Analysis System (RAS)
111 User device 113 User 115 PGS processor 117 PGS I/O interface 119 PGS memory 121 RAS processor 123 RAS I/O interface 125 RAS memory 203 PGS data 205 PGS module 207 First metadata 209 Snapshot data 211 Second Metadata 213 Action sequence data 215 Third metadata 217 Reuse package data 219 Other PGS data 221 Conversion module 223 Metadata generation module 225 Snapshot creation module 227 Sequence construction module 229 Package generation module 231 Other PGS Module 243 RAS Data 245 RAS Module 247 Requirements Data 249 Configuration Data 251 Candidate Reuse Package Data 253 RAS Other Data 255 Receiving Module 257 Configuration Determination Module 259 Reuse Package Determination Module 261 Other RAS Modules 400 Exemplary Computer System 401 I/O Interface of the Example Computer System 402 Processor of the Example Computer System 403 Network Interface 404 Storage Interface 405 Memory of the Example Computer System 406 User Interface 407 Operating System 408 Web Browser 409 Communication Network 411 Input Device 412 Output device

Claims (18)

A method for performing reuse analysis for model lifecycle management, the method comprising:
receiving, by a reuse analysis system (109), input requirements from a user (113) via a user device (111), the input requirements determining a corresponding reuse package from a plurality of reuse packages; the plurality of reuse packages being generated for each of the one or more intermediate steps associated with each of the one or more data processing actions;
determining, by the reuse analysis system (109), a reuse analysis configuration corresponding to a type of reuse package that is appropriate for the input requirements;
the reuse analysis by performing a semantic similarity analysis of the input requirements using one or more pre-stored metadata associated with the plurality of reuse packages based on the reuse analysis configuration; determining, by the system (109), one or more candidate reuse packages from the plurality of reuse packages;
providing the one or more candidate reuse packages as a suggestion to the user by the reuse analysis system (109). 2. The method of claim 1, wherein the input requirements are at least one of business requirements (BR), analysis requirements (AR), and data requirements (DR). Claim: wherein determining the reuse analysis configuration comprises mapping, by the reuse analysis system (109), the input requirements to the types of reuse packages using one or more classification techniques. The method described in Section 1. the one or more candidate reuse packages whose one or more pre-stored metadata semantically match the input requirements by exceeding a preset threshold; 2. The method of claim 1, wherein the method is a package. 5. The one or more candidate reuse packages are assigned a score based on the degree of semantic match between the input requirements and the one or more pre-stored metadata. Method described. Generating the plurality of reusable packages includes:
providing data-as-a-service by converting historical data from a raw format to an easy-to-use format by a package generation system (105) associated with a reusable analysis system (109); first metadata (207) required to generate a reuse package is generated while transforming said historical data, said historical data being received from one or more data sources (101); said providing, stored in a package repository (107) associated with said package generation system (105);
said package generation system (105), while performing said one or more data processing actions, creating one or more state snapshots and providing said data-as-a-service, each of said corresponding data processing actions; constructing an action execution sequence for each of said one or more intermediate steps associated with said action execution sequence, said second metadata (211) for each of said one or more intermediate steps associated with said action execution sequence; said constructing, generated while constructing a sequence;
generating, by the package generation system (105), third metadata (215) for each of the one or more data processing actions, the third metadata (215) comprising: the one or more intermediate steps of the corresponding data processing action, the second metadata (211) generated for each of the one or more intermediate steps, corresponding to the second metadata (211); a reference to the first metadata (207) for each of the one or more intermediate steps, the action execution sequence of each of the one or more intermediate steps, an identifier (ID) corresponding to each action execution sequence, each of the one or more intermediate steps; said generating a predetermined data query, a type of reuse package, and a package identifier (ID) associated with said corresponding data processing action;
The method of claim 1, comprising: generating, by the package generation system (105), the plurality of reuse packages based on the third metadata (215). Providing the data as a service,
parsing, by the package generation system (105), the historical data to learn parameters, constraints, and relationships of the historical data;
creating a directed data dependency state diagram by the package generation system (105) based on the learned parameters, constraints, and relationships of the historical data and a predetermined data navigation schema; creating a directed data dependency state diagram including a data identifier, the historical data, data navigation commands, and data endpoints, each data endpoint being associated with an application programming interface (API);
generating first metadata (207) by the package generation system (105) based on the directed data dependency state diagram and the predetermined data navigation schema in order to provide the data as a service; 7. The method of claim 6, comprising: creating one or more state snapshots and constructing the action execution sequence;
Upon providing the data as a service by the package generation system (105), receiving a data query in the original data format;
converting, by the package generation system (105), the data query from the original data format to a searchable index format representing data endpoints;
providing, by the package generation system (105), the process data corresponding to the data query from each of the data endpoints, along with a storage identifier and location of the process data in a data repository (103); said providing, wherein process data is obtained by reviewing said historical data;
The package generation system (105) determines the state of historical data in the data repository (103), the data query, the process data, and the relationship between the process data and the corresponding data action in which the process data is used. storing relationships thereby creating the one or more state snapshots;
receiving a request from a user (113) via a user device (111) to perform the one or more data processing actions on the process data by the package generation system (105);
creating, by the package generation system (105), a step identifier for each of the one or more intermediate steps associated with each of the one or more data processing actions;
The package generation system (105) generates the second metadata (211) for each of the one or more intermediate steps based on the corresponding data processing action, the step identifier, and the process data. to generate;
constructing the action execution sequence by the package generation system (105) based on the flow defined in the second metadata (211) and a predetermined relationship schema;
7. The method of claim 6, comprising storing the constructed action execution sequence for each request by the package generation system (105). The first metadata (207), the second metadata (211), and the third metadata (215) include pre-stored metadata associated with the plurality of reuse packages. 7. The method of claim 6. A reuse analysis system (109) for performing reuse analysis for model life cycle management, comprising:
a processor (121);
a memory (125) communicatively coupled to the processor (121), wherein the memory (125), upon execution, provides information to the processor;
receiving input requirements from a user (113) via a user device (111), wherein the input requirements facilitate determining a corresponding reuse package from a plurality of reuse packages; , generated for each of the one or more intermediate steps associated with each of the one or more data processing actions;
determining a reuse analysis configuration corresponding to a type of reuse package that is optimal for the input requirements;
the plurality of reuse packages by performing a semantic similarity analysis of the input requirements using one or more pre-stored metadata associated with the plurality of reuse packages based on the reuse analysis configuration; determining one or more candidate reuse packages from the usage packages;
a memory (125) for storing process instructions for providing the one or more candidate reuse packages as a suggestion to the user. The reuse analysis system (109) according to claim 10, wherein the input requirements are at least one of business requirements (BR), analysis requirements (AR), and data requirements (DR). 11. To determine the reuse analysis configuration, the processor is configured to map the input requirements to types of reuse packages using one or more classification techniques. reuse analysis system (109). the one or more candidate reuse packages whose one or more pre-stored metadata semantically match the input requirements by exceeding a preset threshold; The reuse analysis system (109) according to claim 10, which is a package. the processor assigns a score to the one or more candidate reuse packages based on the degree of semantic match between the input requirements and the one or more pre-stored metadata; The reuse analysis system (109) according to claim 13. In order to generate the plurality of reuse packages, a package generation system (105) associated with the reuse analysis system (109),
providing data-as-a-service by converting historical data from a raw format to an easy-to-use format; first metadata (207) required for generating said plurality of reuse packages; is generated while transforming said historical data, said historical data being received from one or more data sources (101) and stored in a data repository (103) associated with said package generation system (105); providing said;
creating one or more state snapshots while performing the one or more data processing actions and providing the data-as-a-service, the one or more intermediates associated with each of the corresponding data processing actions; constructing an action execution sequence for each of the steps, wherein second metadata (211) for each of the one or more intermediate systems is generated while constructing the action execution sequence; to build,
generating third metadata (215) for each of the one or more data processing actions, wherein the third metadata (215) includes the corresponding one of the corresponding data processing actions; The above intermediate step, the second metadata (211) generated for each of the one or more intermediate steps, and the first metadata (207) corresponding to the second metadata (211). ), the action execution sequence of each of the one or more intermediate steps, an identifier (ID) corresponding to each action execution sequence, an identifier (ID) associated with each of the one or more intermediate steps and the corresponding data processing action. the plurality of reuse packages based on the third metadata (215); and generating the plurality of reuse packages based on the third metadata (215). A reuse analysis system (109) according to claim 10, configured to perform. In order to provide the data as a service, the package generation system (105)
parsing the historical data to learn parameters, constraints, and relationships of the historical data;
creating a directed data dependency state diagram based on the learned parameters, constraints, and relationships of the historical data and a predetermined data navigation schema, wherein the directed data dependency state diagram creating an identifier, the historical data, a data navigation command, and a data endpoint, each data endpoint being associated with an application programming interface (API);
Claim: configured to generate the first metadata (207) based on the directed data dependency state diagram and the predetermined data navigation schema for providing the data as a service. Reuse analysis system (109) according to item 15. the package generation system (105) for creating one or more state snapshots and constructing the action execution sequence;
providing said data-as-a-service, receiving data queries in their original data format;
converting the data query from the original data format to a searchable index format representing data endpoints;
providing said process data corresponding to said data query from each said data endpoint, along with a storage identifier and location of said process data in said data repository (103), said process data detailing said historical data; obtained by considering, providing;
storing the state of historical data in the data repository (103), the data queries, the process data, and the relationships between the process data and the corresponding data actions in which the process data is used; creating and storing the one or more state snapshots;
receiving a request from a user (113) via a user device (111) to perform the one or more data processing actions on the process data;
creating a step identifier for each of the one or more intermediate steps involved in each of the one or more data processing actions;
generating said second metadata (211) for each of said one or more intermediate steps based on said corresponding data processing action, said step identifier and said process data;
constructing the action execution sequence based on the flow defined in the second metadata (211) and a predetermined relationship schema; and storing the constructed action execution sequence for each request. A reuse analysis system (109) according to claim 15, configured to perform. The first metadata (207), the second metadata (211), and the third metadata (215) include pre-stored metadata associated with the plurality of reuse packages. A reuse analysis system (109) according to claim 15, which forms a reuse analysis system (109).

Images