WO2015171117A1 - Backend analysis and transformation - Google Patents

Backend analysis and transformation Download PDF

Info

Publication number
WO2015171117A1
WO2015171117A1 PCT/US2014/036945 US2014036945W WO2015171117A1 WO 2015171117 A1 WO2015171117 A1 WO 2015171117A1 US 2014036945 W US2014036945 W US 2014036945W WO 2015171117 A1 WO2015171117 A1 WO 2015171117A1
Authority
WO
WIPO (PCT)
Prior art keywords
selected
transformation
definition
input data
format
Prior art date
Application number
PCT/US2014/036945
Other languages
French (fr)
Inventor
Jean GUIANNOTTE
Oleksandr SUDARSKYY
Sean LENNON
Original Assignee
Hewlett-Packard Development Company, L.P.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hewlett-Packard Development Company, L.P. filed Critical Hewlett-Packard Development Company, L.P.
Priority to PCT/US2014/036945 priority Critical patent/WO2015171117A1/en
Publication of WO2015171117A1 publication Critical patent/WO2015171117A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/448Execution paradigms, e.g. implementations of programming paradigms
    • G06F9/4488Object-oriented
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/60Software deployment
    • G06F8/61Installation
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45504Abstract machines for programme code execution, e.g. Java virtual machine [JVM], interpreters, emulators

Abstract

According to an example, backend analysis and transformation may include receiving input data that includes a format that is specific to an input source, and selecting a transformation from a plurality of available transformations to transform the input data to an output including a format that is specific to an output source. The input data may be transformed to generate the output based on the selected transformation that includes a definition archive that is selected from a plurality of available definition archives to drive the selected transformation, and external packages that are selected from a plurality of available external packages to implement the selected definition archive. The selected definition archive and external packages are hot-pluggable at runtime to dynamically allow new solutions related to the selected transformation to be implemented.

Description

BACKEND ANALYSIS AND TRANSFORMATION

BACKGROUND

[0001] Data transformation typically converts a set of data values from the data format of a source data system into the data format of a destination data system. Data transformation typically includes a data mapping process that maps data elements from the source data system to the destination data system, and captures any transformation that is to occur. Data transformation typically also includes generation of machine readable instructions that create the actual transformation application to convert the set of data values from the data format of the source data system into the data format of the destination data system.

BRIEF DESCRIPTION OF DRAWINGS

[0002] Features of the present disclosure are illustrated by way of example and not limited in the following figure(s), in which like numerals indicate like elements, in which:

[0003] Figure 1 illustrates an architecture of a backend analysis and

transformation apparatus, according to an example of the present disclosure;

[0004] Figure 2 illustrates loaders for the backend analysis and transformation apparatus of Figure 1 , according to an example of the present disclosure;

[0005] Figure 3 illustrates services for the backend analysis and transformation apparatus of Figure 1 , according to an example of the present disclosure;

[0006] Figure 4 illustrates an execution diagram for the backend analysis and transformation apparatus of Figure 1 , according to an example of the present disclosure;

[0007] Figure 5 illustrates a method for backend analysis and transformation, according to an example of the present disclosure;

[0008] Figure 6 illustrates further details of the method for backend analysis and transformation, according to an example of the present disclosure; and

[0009] Figure 7 illustrates a computer system, according to an example of the present disclosure.

DETAILED DESCRIPTION

[0010] For simplicity and illustrative purposes, the present disclosure is described by referring mainly to examples. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be readily apparent however, that the present disclosure may be practiced without limitation to these specific details. In other instances, some methods and structures have not been described in detail so as not to unnecessarily obscure the present disclosure.

[0011] Throughout the present disclosure, the terms "a" and "an" are intended to denote at least one of a particular element. As used herein, the term "includes" means includes but not limited to, the term "including" means including but not limited to. The term "based on" means based at least in part on.

[0012] Industries that use automated analysis and reporting solutions are often characterized by fast changing business needs and emerging new technologies. As such, having maximum flexibility typically facilitates success. Analysis services using collected data span domains such as server configuration, product firmware, and security. Initiatives to increase agility and to reduce the cost and time to market of new services, coupled with information technology's (IT's) centralized data strategy have created the conditions where a new approach to service creation has emerged. These conditions and similarities between services suggest that use of components as reusable building blocks may be beneficial in this problem domain.

[0013] Creation of new assessments in practice means responding to frequent input data changes, collaboration with the business users on what the report output should look like, and many iterations in between. The typical approach of parsing and loading data into Java containers needs code changes when the incoming data changed. These aspects may distract developers from the transformation of the data itself, and do not give developers a view of the data they are transforming. [0014] In this regard, according to examples, a backend analysis and

transformation apparatus and a method for backend analysis and transformation are disclosed herein. The apparatus and method disclosed herein provide for new solutions using configurations of reusable components. At their core, the apparatus and method disclosed herein include hot-pluggability capability that allows configuration packages including definitions, scripts, loaders, and other executables to be injected therein. This is driven by a definition archive package that provides a complete end to end solution. The framework for the apparatus and method disclosed herein may allow definition of a hierarchical transformation flow and control of its execution.

[0015] The apparatus and method disclosed herein may filter or transform input data into a new output representation which may take a plurality of forms.

Generally, the apparatus and method disclosed herein provide a generic machine readable instructions based engine that includes hot-pluggability at its core that includes a transformation module, with a set of readily implemented adapters, loaders, services, and processes that are targeted at the data transformation and data integration problem space. The apparatus and method disclosed herein may be polymorphic in multiple dimensions that include an execution environment, data flow control, solution logic, and solution components. Designed to allow rapid development of new data analysis and reporting solutions, the apparatus and method disclosed herein may combine Java technologies such as dependency injection, reflection, in-memory databases, object relational mapping (ORM), parallel data processing, and Extensible Markup Language (XML). A definition package payload (i.e., the definition archive as described herein) for the apparatus and method disclosed herein may include executable instructions and resources. This definition archive may drive the transformation of the input data for the apparatus and method disclosed herein, and is hot-pluggable at runtime to allow new solutions related to the apparatus and method disclosed herein to be deployed dynamically without a full software (i.e., machine readable instructions) or execution framework software release. For example, an updated executable payload may be pulled from a database or a repository. The apparatus and method disclosed herein may be applied to a variety of different technology areas. For example, the apparatus and method disclosed herein may be used as an integration engine that combines data from multiple data sources to generate an output in a predetermined format, or as a dynamic consumer/producer to any service orientated architecture web service.

[0016] With respect to the hot-pluggability aspect for the apparatus and method disclosed herein, hardware based hot swapping typically describes replacing components without significant interruption to a system, while hot-plugging describes the addition of components to expand the system without an extensive interruption to the operation of the system. Thus, a user may plug and unplug a component without restarting an application, or rebooting of machine readable instructions. Machine readable instructions based hot-pluggability may be considered as the ability of a framework to accept payload or execution content (e.g., a set of computation instructions, binary code, scripts, resources, etc.) as a bundle that will dynamically modify the behavior of the framework without restart.

[0017] As described herein, the apparatus and method disclosed herein may be polymorphic in multiple dimensions that include an execution environment, data flow control, solution logic, and solution components. According to an example, the execution environment for the apparatus and method disclosed herein is a Java virtual machine (JVM) that is a combination of libraries, resources, and JVM settings available to the JVM at runtime. With respect to data flow control, the apparatus and method disclosed herein may control the topology of how the execution occurs. This includes control over serial versus parallel executions of solution components. The solution logic for the apparatus and method disclosed herein may be included within a bundle payload, denoted the definition archive, and includes component wiring instructions (including conditional wirings), scripts, data loaders instructions, transformation sections, and execution of services instructions. The definition archive for the apparatus and method disclosed herein may include solution components such as compiled code, third party Java libraries, predefined reusable syntax wirings, and other miscellaneous resources to provide for execution of the apparatus and method disclosed herein.

[0018] The apparatus and method disclosed herein may include modular architecture based on run-time hot-pluggable components definitions loaded from a definition archive, and libraries and resources packages loaded from external bundle packages. The apparatus and method disclosed herein may include flexibility, for example, to reuse, connect, and integrate existing components to build new solutions, a unique set of readily wired adapters, loaders, services, and processes targeted at the data transformation and data integration problem space, and utilization of new emergent technologies, business services, or changes to business services that are dynamically plugged in at run-time. The apparatus and method disclosed herein may include modularity, for example, to build modular solutions, abstract solutions from external dependencies, and deploy the modular solutions in a hot-pluggable manner. The apparatus and method disclosed herein may provide for separation of concerns, for example, to perform analysis, integration, aggregation, and reporting on input data from various sources in an efficient manner. The apparatus and method disclosed herein may reduce the need for framework dependency, for example, by providing hot-pluggability, transformation flow capacity, and parallelization and serialization control on top of any execution framework (e.g., cloud, local personal computer (PC) JVM, etc.). The apparatus and method disclosed herein may provide for extensibility, for example, based on the ability to take advantage of new technologies (e.g. any Java library functionalities), and extension of functionality through reusable plugins. The apparatus and method disclosed herein may provide for maintainability, for example, to evolve rapidly, and to reduce technology decay. The apparatus and method disclosed herein may address the need for faster time to market and reduce the cost of development, for example, by providing the ability to build and deploy new solutions readily by reusing, connecting, and integrating existing components, and by providing scalability. The apparatus and method disclosed herein may be executed in a scalable cloud environment across multiple nodes and use parallelization in its architecture, or execute on a single device (e.g., a PC).

[0019] The apparatus and method disclosed herein may provide a uniformly organized framework for development of end to end solutions. As described herein, a step-based approach may be used for development of new

transformations.

[0020] The apparatus and method disclosed herein may be highly flexible in its deployment. For example, the apparatus and method disclosed herein may execute in their own execution context environment, and the definition archive may be hot-plugged dynamically at run-time, thus providing an environmental polymorphic quality. Further, parameters of the framework may be passed to the apparatus and method disclosed herein. This aspect may decouple the framework from the solution (definition) execution.

[0021] For the apparatus and method disclosed herein, different definition archives (i.e., definition compressed zip packages) may be fed from a repository at run time, which may result to different behaviors as needed. Additionally, environmental parameters (e.g., integration environment set of parameters and/or property values for wired placeholders resolution, production environment set of parameters and/or property values for wired placeholders resolution, database connection strings, data sources) and configuration parameters (e.g., debug level, amount of memory to use, number of operational threads, directories to use, whether an internal database for the apparatus runs in-memory or not, and pointers to Hypertext Transfer Protocol (HTTP) located resources) may be configured dynamically.

[0022] For the apparatus and method disclosed herein, solution flow may be tuned for optimum performance for a specific execution environment without change of the associated machine readable instructions, and restarting of the execution framework. Control over settings for parallelization, serialization, limiting flow capacity, exception handling, conditional execution abort may be implemented in a dynamic manner. This elastic behavior allows the apparatus and method disclosed herein to respond to changing environmental conditions, and adapt behavior to different load conditions.

[0023] The apparatus and method disclosed herein may be executed in any environment that executes a JVM. The apparatus and method disclosed herein may be executed from a personal computer (PC) to a cloud environment. The apparatus and method disclosed herein may include a uniform execution of definitions that are controlled based on launcher properties that facilitate execution. The apparatus and method disclosed herein may provide a consistent mechanism of execution through the launcher which facilitates execution in any environment, such as, for example, a cloud environment, or a PC environment.

[0024] The apparatus and method disclosed herein may include an internal database that is used to transfer input data, and to iterate over transformations to facilitate visualization and querying of data to provide an improved understanding of the data structure. The internal database may toggle operation as an in-memory database, or as a file system database. The internal database may be an internal relational database.

[0025] For the apparatus and method disclosed herein, approved solutions, enhancements, content upgrades, and defect fixes may be individually hot- deployed (i.e., without core framework release) to test, integration, or production environments.

[0026] The apparatus and method disclosed herein may embed parallelization into its core. Thus, the apparatus and method disclosed herein may include a chain of execution stages, with any stage in the chain of execution stages being executable in parallel as its default behavior. For example, if a large data set is to be loaded, the loader stage may be configured to execute in parallel without limit, except for limits imposed by the execution environment. [0027] The apparatus and method disclosed herein may also allow third party Java libraries to be plugged in as needed at runtime.

[0028] Figure 1 illustrates an architecture of a backend analysis and

transformation apparatus (hereinafter also referred to as "apparatus 100"), according to an example of the present disclosure. Referring to Figure 1 , the apparatus 100 is depicted as including a transformation module 102 that includes an object factory 104, a wiring interpreter module 106, a plugins class loader 108, an Analysis Transformation Language (ATL) interpreter module 110, a Structured Query Language (SQL) engine 112, a dynamic Object-Relational Mapping (ORM) module 114, and a launcher 1 16, the operation of each of which is described in further detail herein. The apparatus 100 may further include a generic components module 118 that includes generic data loaders 120, generic commands 122, and generic data services 124, the operation of each of which is described in further detail herein. The apparatus 100 may receive input data 126 that is in the form of a file, data from a database, a web service, etc. The input data 126 may be loaded into an internal relational database 128. The apparatus 100 may transform the input data 126 into output 130 for output to another file, another database, another web service, etc. Thus, the apparatus 100 may operate as an adapter, or provide for the implementation of an adapter, to transform the input data 126 from a format specific to an input source to another format specific to an output source. The output 130 may be further transformed or consumed by another service as needed. The output 130 of the apparatus 100 is not restricted to the direct output of the apparatus 100. The apparatus 100 may produce multiple transformation outputs depending on the services configured in a definition archive 132 that the apparatus 100 runs (i.e., output of reporting documents in various formats, output of the transformed data into various connected destination data sources, calling various external web or File Transfer Protocol (FTP) services by providing routing or transformation by the apparatus 100, etc.). The apparatus 100 may also be used as any framework module that separates the execution framework from code dependencies by wrapping the code into the definition archive 132 and executing the code under full control of the apparatus 100. In this case the output 130 of apparatus 100 may include the protocol of this code execution and any errors that have occurred. This code may be made hot-pluggable to the execution framework. The output 130 may be used for the purposes of monitoring execution of this wrapped code. The definition archive 132 may drive the transformation of the input data 126, and is hot-pluggable at runtime to allow new solutions related to the apparatus 100 to be deployed dynamically without a driving framework machine readable instructions based release. The definition archive 132 may include a free structure and may include a model 134, wiring instructions 136, compiled and scripting code 138, SQL scripts 140, ATL sections 142, resources 144, plugins 146, and libraries 148, the operation of each of which is described in further detail herein. External packages 150 may be used by the apparatus 100 and participate in the definition archive 132 solution dependencies. The external packages 150 may include Java bundles of machine readable instructions, and resources such as, for example, Web Archive (WAR), Enterprise Archive (EAR), Java Archive (JAR), or non-standard ZIP compressed bundles.

[0029] The modules and other elements of the apparatus 100 may be machine readable instructions stored on a non-transitory computer readable medium. In this regard, the apparatus 100 may include or be a non-transitory computer readable medium. In addition, or alternatively, the modules and other elements of the apparatus 100 may be hardware or a combination of machine readable instructions and hardware.

[0030] The transformation module 102 may include the object factory 104, the wiring interpreter module 106, the plugins class loader 108, the ATL interpreter module 110, the SQL engine 112, the dynamic ORM module 114, and the launcher 116.

[0031] The object factory 104 may be used to wire and instantiate objects inside the apparatus 100 instance, as well as to transport externally wired objects that are passed to the apparatus 100 as the parameters thereof. For example, the object factory 104 may be used to wire and instantiate service beans or any other Java beans inside the apparatus 100 instance, as well as to transport the externally wired beans that are passed to the apparatus 100 as the parameters thereof. The instantiated objects within apparatus 100 may be used on any level of a

transformation (e.g., loaders, commands, services, etc.). The apparatus 100 may connect to an input data source by configuring a wiring bean pointing at the data source. An appropriate loader (e.g., one of the generic data loaders 120) may transfer the data into the internal relational database 128. Similarly an output data source may be constructed, and transformed data may be routed to the output data source as the output 130.

[0032] The wiring interpreter module 106 may process object wiring instructions such as, for example, factory beans, context objects, and conditional beans. The module 106 may also process "include" wiring instructions to aggregate the separate wiring modules (separate files including factory beans, context objects, and conditional beans wirings), located elsewhere within the definition archive 132, into common wiring context.

[0033] The plugins class loader 108 may be a Java uniform resource locator (URL) classloader extension used by the apparatus 100. The implementation of the plugins class loader 108 may locate and load classes and resources from the definition archive 132, and delegate the searches to the classloader's chain. The plugins class loader 108 may load external packages 150 and the contents

(classes and resources) of the definition archive 132 under the same

transformation environment. Thus, the plugins class loader 108 may construct the common (merged) apparatus 100 transformation environment built from external packages 150 and the contents of the definition archive 132.

[0034] The ATL interpreter module 110 may include markup based Analysis and Transformation Language (ATL) instructions to define and drive the hierarchical transformation flow for the apparatus 100. The ATL module 110 may act on top of the internal relational database 128 by exploiting the apparatus 100 internal SQL engine 112. The ATL interpreter module 110 may also control transformation flow capacity, parallelization, and serialization.

[0035] The SQL engine 112 may use a relational database SQL engine which may operate in-memory, or in file system mode while using the internal relational database 128.

[0036] The dynamic ORM module 114 may persist any interlinked Plain Old Java Object (POJO) structures into the internal relational database 128 without any additional specific configurations being needed. In other words, the dynamic ORM module 114 may persist into a SQL query-able form any interlinked POJO structures into the internal relational database 128 while preserving data relation links to implement the selected transformation. POJO structures may represent Java objects that do not follow any of the major Java object models, conventions, or frameworks. The dynamic ORM module 1 14 may dynamically generate the relational structure (tables, indexes, integrity links) inside the internal relational database 128 by inspecting passed POJO Java classes composition ("has-a") and hierarchy ("is-a"), as well as map and persist the POJO objects values into this created relational structure. The dynamic ORM module 114 may be used within the data loader implementation of the generic data loaders 120.

[0037] The launcher 116 may provide for the apparatus 100 to be dynamically configured from a set of defined properties, and launched in the same JVM instance, or alternatively, a new separate JVM instance (separate process) may be spawned.

[0038] The generic components module 118 may include the generic data loaders 120, the generic commands 122, and the generic data services 124.

[0039] The generic data loaders 120 may load data into the internal relational database 128 (i.e., the runtime database). The generic data loaders 120 may not be restricted by any specific high level data structure limitations of the data to be loaded, but instead, may provide the generic mechanisms to interface with underlying technologies on which the data is based, sourced, or obtained. The generic data loaders 120 may be executed under the control of the apparatus 100. The generic data loaders 120 may be configurable, and implement a variety of data loading tasks. Examples of the generic data loaders 120 are shown in Figure 2. Referring to Figure 2, the generic data loaders 120 may include a

RunlnstanceLoader, a BeanLoader, a CallRouterLoader, a ClassStructureLoader, a DataSourceLoader, a GroovyLoader, a JaxbXmlLoader, an

ORMPojoGenericLoader, a MarshalTOutputLoader, a PojoLoader, a

RegexpLoader, a SQLScriptLoader, a TransformTOutputLoader, an

XPathGenericLoader, an XPathGenericMultiResourceLoader, and an

XPathGenericTransformLoader. The generic data loaders 120 may operate in conjunction with wiring instructions located in the definition archive 132 to deliver a variety of data to the apparatus 100 for further transformation.

[0040] The RunlnstanceLoader may execute any loader implementing ILoader interface passed as a parameter.

[0041] The BeanLoader is an ORM loader that may load an object hierarchy returned by any bean's method call. The BeanLoader may load the object factory 104 registered bean or the returned result object of a bean's method invocation into the internal relational database 128 using the dynamic ORM module 114.

[0042] The CallRouterLoader may call a method of a factory bean.

[0043] The ClassStructureLoader is an ORM loader that may create a database tables structure from a list of classes or beans. The ClassStructureLoader may dynamically build the hierarchical interlinked tables structures in the internal relational database 128 (or in an external database) based on the Java class composition ("has-a") and hierarchy ("is-a") of passed classes, objects, or factory beans. The ClassStructureLoader may thus operate without any prior knowledge of a database structure. [0044] The DataSourceLoader may use a JDBC (Java Database Connectivity) data source (javax.sql.DataSource), wired as an object factory 104 bean, or constructed as an inline loader's parameter object, to perform a SQL query against this data source, and load a query execution result into the internal relational database 128.

[0045] The GroovyLoader is a proxy loader for Groovy script. The

GroovyLoader may execute a Groovy language script, and load the returned result object into the internal relational database 128 by using the dynamic ORM module 144.

[0046] The JaxbXmlLoader is an ORM loader for loading Java Architecture for XML Binding (JAXB) objects by un-marshaling, and transforming an XML document obtained from an input source into the internal relational database 128.

[0047] The ORMPojoGenericLoader may provide base functionality which is common to the loaders that are using the dynamic ORM module 114 capabilities for the apparatus 100. The ORMPojoGenericLoader may not depend on any specific parameter, and may be used as a base class for the other loaders for the

apparatus 100, or for ORM loaders.

[0048] The MarshalTOutputLoader may marshal the output of apparatus 100 to an XML file.

[0049] The PojoLoader is an ORM Java POJO loader that uses the dynamic ORM module 114. The PojoLoader may dynamically build the hierarchical interlinked tables structures in the internal relational database 128 based on the Java class composition ("has-a") and hierarchy ("is-a") of passed object's class, and populate created structure with objects content. The composition and hierarchy of POJO class may be inspected, as well as all needed object content to table mappings may occur at runtime. Thus, the PojoLoader may operate without any prior knowledge of the structure of the internal relational database 128.

[0050] The RegexpLoader is a generic loader that may parse the textual data content with specified regular expressions (RegExp), and persist the result of parsing to the internal relational database 128.

[0051] The SQLScriptLoader is a generic SQL script loader that may execute the compatible SQL statements script of the SQL engine 112 against the internal relational database 128. The script may be returned, in a general case, by a bean method invocation.

[0052] The TransformTOutputLoader may marshal and Extensible Stylesheet Language Transformation (XSLT) transform the direct output (TOutput) of the apparatus 100 into a file designated by the relative path.

[0053] The XPathGenericLoader may allow for loading of XML Document Object Model (DOM) trees that are the results of the XPath queries against an XML document, into the internal relational database 128.

[0054] The XPathGenericMultiResourceLoader is an XPath query XML DOM based loader. The XPathGenericMultiResourceLoader may allow for loading of XML DOM trees that are results of multiple XPath queries against an XML document into the internal relational database 128.

[0055] The XPathGenericTransformLoader is an XPath query XML DOM based loader with additional XSLT transformation capabilities.

[0056] The generic commands 122 may include machine readable instructions that are executed during the ATL transformation phase by using the ATL command instructions. The ATL command instructions are SQL instructions embedded into an XML schema. According to an example, the generic commands 122 may include Java or Groovy machine readable instructions.

[0057] The generic data services 124 may include the machine readable instructions that are registered and executed by the apparatus 100 as defined by the model 134. The generic data services 124 may consume and process processing instructions created during the ATL transformation phase. Each processing instruction for a given service may be linked to an action that the service is supposed to act upon, as well as an instruction may include parameters linked to the action that will be passed to a service. Hence, the service may act on processing instructions. The internal relational database 128 with the data loaded for transformation is available to the generic data services 124 (as generic services, as well as custom built services that may be included as a part of the definition archive 132 plugins, or be a part of the external packages 150) through any available SQL engine 112 communication interfaces.

[0058] Figure 3 illustrates generic data services 124 for the apparatus 100, according to an example of the present disclosure. The generic data services 124 may include MarshalTOutputService, TransformTOutputService,

OutputModelService, and ZipCompressService.

[0059] The MarshalTOutputService may marshal the direct output (TOutput) of the apparatus 100 into an XML file designated by the relative path that is specified.

[0060] The TransformTOutputService may transform the direct output (TOutput) of the apparatus 100 to a file using an XSLT specified. A default XSLT may also be provided.

[0061] The OutputModelService may hierarchically output the database level object's data from the internal relational database 128 structure into a generic set of Java Objects.

[0062] The ZipCompressService may compress specified directories or files relative to a work directory for the apparatus 100 into a ZIP compressed archive.

[0063] Referring to Figure 1 , the definition archive 132 of the apparatus 100 may be hot-pluggable into the apparatus 100. Further, any Java bundle may also be injected into the apparatus 100, and utilized by the apparatus 100. The apparatus 100 may construct a chain of class loaders, and merge environments together at runtime. In the configuration wiring of the apparatus 100, references may be made to the external packages 150. Generally, any resources that are needed for the hot-pluggability may be included in the definition archive 132. Thus, the definition archive 132 may be hot-pluggable at runtime to allow new solutions for the apparatus 100 to be deployed dynamically without a release of a particular set of machine readable instructions. The definition archive 132 may thus provide for the apparatus 100 to be polymorphic in multiple dimensions based on the specifics of the transformation.

[0064] The definition archive 132 is a ZIP compressed bundle, and may include the model 134, the wiring instructions 136, the compiled and scripting code 138, the SQL scripts 140, the ATL sections 142, the resources 144, the plugins 146, and the libraries 148. The internal structure and the content of the definition archive 132 is not restricted as long as the relative path references pointing to the resources located within the definition archive 132 may be resolved at runtime when the solution is executed by the apparatus 100. The model 134 may be located at the root of the definition archive 132.

[0065] The model 134 may include a model definition file. For example, the model 134 includes a model-definition.xml file. The XML file may define the complete end to end transformation flow that will be performed by the apparatus 100. For the model 134, at the beginning of the execution of a transformation, the apparatus 100 may pass the initial control of any execution of a transformation to the model 134. The model 134 may define the framework structure of the transformation flow, and operate as the connection (aggregation) point of all of the other transformation components for the definition archive 132. For example, the model 134 may connect inclusions of bean wiring blocks, loaders, SQL scripts, ATL sections, services, and provide control instructions for parallelization and/or serialization of the transformation flow.

[0066] The wiring instructions 136 for the definition archive 132 may be used by the apparatus 100, for example, for Java beans wiring and conditional beans wiring at different phases of a transformation. The conditional beans wiring may provide for the dynamic wiring of beans based on the value of one or more other beans. Using this data driven approach, the wiring path may be dynamically chosen depending on the outcome of previous wirings. For example, if bean A == bean B then wire bean C (or go to the wiring path C) else wire bean D (or go to the wiring path D).

[0067] The compiled and scripting code 138 for the definition archive 132 may include any Java or Groovy code loaded by the apparatus 100 at runtime from the definition archive 132.

[0068] The SQL scripts 140 for the definition archive 132 may be used to define database objects (e.g., data definition language (DDL) statements) or manipulate data (e.g., data manipulation language (DML) statements).

[0069] The ATL sections 142 may include XML documents that include the ATL instructions. Complex data transformation flows may be defined with the ATL syntax. ATL is a mark-up based language to define and drive the hierarchical transformation flow designed for the apparatus 100. The ATL instructions may be considered as higher level instructions than the SQL engine 112 instructions acting on the internal relational database 128 and the object factory 104 for the apparatus 100. Generally, the ATL for the ATL sections 142 may be used to create a set of processing instructions based on the loaded input data 126. The processing instructions may be embedded with output data for the output 130, and allow post processing to occur on this output data. Processing instruction may include a target service name (e.g. MyReportService), an action (e.g., CreateReport), and a set of parameters (e.g., reportTemplate=/templates/birt/myReport.rptdesign;

outputFormat=PDF; outputFile=/reports/pdf/myReport.pdf). The processing instructions may drive the services that run subsequently during the transformation. The ATL syntax for the ATL sections 142 may provide facilities for processing instructions generation, and further, the ATL syntax may be used for constructing rich presentation outputs.

[0070] Resources 144 may include any binary or textual files or configurations used by the transformation that may be pulled from the definition archive 132 as requested. Resources 144 may also include images, templates, etc., that are used for the transformation.

[0071] Plugins 146 may include Java libraries that are packaged under the plugins or library directories of the definition archive 132, and loaded by the plugins class loader 108. The plugins 146 are the core extensions or implementations of a loader, command, or service for the apparatus 100.

[0072] Libraries 148 may include any third party Java libraries that the apparatus 100 transformation solution in a form of the definition archive 132 depends upon. These libraries may be dynamically loaded during transformation of the input data 126 to the output 130.

[0073] The external packages 150 may include Java bundles of machine readable instructions, and resources such as, for example, WAR, EAR, JAR, or non-standard ZIP compressed bundles. A JAR file includes a Jar extension for Java application. An EAR file includes an .ear extension for Java, Java Servlets, Java Server Pages (JSP), and Enterprise JavaBeans (EJB). A WAR file includes a .war extension for Java, and Java Servlets and JSP. JAR is a platform- independent file format that aggregates many files into one. A JAR file may include the class files and auxiliary resources such as library files, properties file, image, sound, etc. The WAR file may be deployed in a web application, and include servlets, JSPs, and their supporting classes and files. A servlet container may be used to deploy these files, and includes files and directories within a WAR file. A WAR file may include a specific directory structure. The top-level directory of a WAR is the document root (WEB-INF) of the application. The document root is where JSP pages, client-side classes and archives, and static Web resources are stored. WEB-INF may include web.xml, classes, lib, and Tag library descriptor files. An EAR file is a JAR file with an .ear extension which is generally a Java 2 Platform, Enterprise Edition (J2EE) application. The EAR file may include entries representing the modules of an application, and a metadata directory called META- INF which includes one or more deployment descriptors. The EAR file is a top- level container that includes modules, such as, EJB modules, web modules, application client modules, etc., and deployed to an application server, such as WebLogic, WebSphere, JBOSS, etc. The EAR file may include WARs, JARs, etc.

[0074] The external packages 150 may also be hot-pluggable at runtime to allow new solutions for the apparatus 100 to be deployed dynamically without a release of a particular set of machine readable instructions. The same set of external packages 150 may be used by different definition archives 132, or different sets of external packages 150 may be used by different definition archives 132 as needed based on the specifics of a transformation by apparatus 100.

[0075] Figure 4 illustrates an execution diagram 400 for the apparatus 100, according to an example of the present disclosure. With respect to the framework 402, the apparatus 100 may execute independently from the framework 402. The framework 402 may execute the launcher 116 of the apparatus 100 (note, the launcher 116 is shown separately from the apparatus 100 for facilitating a description of the execution diagram 400). The framework 402 may represent, for example, a cloud environment, a JVM running on a local PC, Apache Tomcat, Eclipse development environment, etc. The apparatus 100 may also reduce the need for framework operational load and dependencies, for example, by providing hot-pluggability, transformation flow capacity, and parallelization and serialization control on top of the framework 402 at runtime when a specific definition archive 132 executes. Different definition archives 132 (e.g., definition archive #1 , definition archive #2, etc.) may be used to implement different transformations for the apparatus 100. Similarly, different packages 150 (e.g., package #1 , package #2, etc.) may be input to the launcher 116, and provide the bundles of libraries that are needed for the different definition archives 132 to execute. Environment configuration 404 and launcher configuration 406 may also be provided to the launcher 116, and used by the model 134 of the definition archives 132. The launcher 116 may execute the apparatus 100. The launcher configuration 406 may define which of the definition archives 132 are needed for a particular

transformation. The particular definition of the definition archives 132 that is pulled by the apparatus 100 may then be used to drive the particular transformation. The output 130 may include the particular transformation output, and also any instructions for further processing of the particular transformation.

[0076] Figures 5 and 6 respectively illustrate flowcharts of methods 500 and 600 for backend analysis and transformation, corresponding to the example of the backend analysis and transformation apparatus 100 whose construction is described in detail above. The methods 500 and 600 may be implemented on the backend analysis and transformation apparatus 100 with reference to Figures 1 -4 by way of example and not limitation. The methods 500 and 600 may be practiced in other apparatus.

[0077] Referring to Figure 5, for the method 500, at block 502, the method may include receiving input data that includes a format that is specific to an input source. For example, referring to Figure 1 , the transformation module 102 may receive the input data 126 that includes a format that is specific to an input source.

[0078] At block 504, the method may include selecting a transformation from a plurality of available transformations to transform the input data to an output including a format that is specific to an output source. For example, referring to Figures 1 and 4, the transformation module 102 may select a transformation from a plurality of available transformations to transform the input data to the output 130 including a format that is specific to an output source.

[0079] At block 506, the method may include transforming the input data to generate the output including the format that is specific to the output source based on the selected transformation. For example, referring to Figures 1 and 4, the transformation module 102 may transform the input data 126 to generate the output 130 including the format that is specific to the output source based on the selected transformation. The selected transformation may use a definition archive that is selected from a plurality of available definition archives to drive the selected transformation of the input data to generate the output including the format that is specific to the output source. For example, referring to Figures 1 and 4, the selected transformation may use a definition archive 132 that is selected from a plurality of available definition archives (e.g., definition archive #1 , definition archive #2, etc.) to drive the selected transformation of the input data 126 to generate the output 130 including the format that is specific to the output source. The selected definition archive includes a set of instructions that are hot-pluggable at runtime to dynamically allow new solutions related to the selected transformation to be implemented. For example, referring to Figures 1 and 4, the selected definition archive 132 includes a set of instructions (e.g., the instructions of the components of the definition archive 132) that are hot-pluggable at runtime to dynamically allow new solutions related to the selected transformation to be implemented.

[0080] Referring to Figure 6, for the method 600, at block 602, the method may include receiving input data that includes a format that is specific to an input source.

[0081] At block 604, the method may include selecting a transformation from a plurality of available transformations to transform the input data to an output including a format that is specific to an output source.

[0082] At block 604, the method may include transforming the input data to generate the output including the format that is specific to the output source based on the selected transformation. The selected transformation may use a definition archive that is selected from a plurality of available definition archives to drive the selected transformation of the input data to generate the output including the format that is specific to the output source. The selected definition archive may include a set of instructions that are hot-pluggable at runtime to dynamically allow new solutions related to the selected transformation to be implemented. Further, the selected transformation may use external packages (e.g., the external packages 150) that are selected from a plurality of available external packages (e.g., see Figure 4) to implement the selected definition archive. The selected external packages may be hot-pluggable at runtime to implement the selected definition archive, and the plurality of available external packages include library bundles (e.g., EAR, WAR, JAR, etc.) to implement different ones of the plurality of available definition archives.

[0083] According to an example, transforming the input data to generate the output including the format that is specific to the output source based on the selected transformation may further include performing the selected transformation in JVM execution environment.

[0084] According to an example, transforming the input data to generate the output including the format that is specific to the output source based on the selected transformation may further include controlling serial and parallel execution of machine readable instructions (e.g., by the transformation module 102) related to the selected transformation.

[0085] According to an example, the selected definition archive may include compiled code (e.g., the compiled and scripting code 138), third party Java libraries (e.g., the libraries 148), and predefined reusable syntax wirings (e.g., the wiring instructions 136) to implement the selected transformation.

[0086] According to an example, transforming the input data to generate the output including the format that is specific to the output source based on the selected transformation may further include performing the selected transformation independently of an execution framework (e.g., the framework 402) for the selected transformation.

[0087] According to an example, transforming the input data to generate the output including the format that is specific to the output source based on the selected transformation may further include using generic data loaders (e.g., the generic data loaders 120) to load the input data 126. The generic data loaders may include a class structure loader (e.g., ClassStructureLoader of Figure 2) that is an ORM loader to create a database table structure from a list of classes related to the input data to implement the selected transformation.

[0088] According to an example, the selected definition archive includes a model definition XML file (e.g., the model 134) that defines a complete

transformation flow for the selected transformation.

[0089] According to an example, the selected external packages include WAR, EAR, and JAR files to implement the selected definition archive.

[0090] Figure 7 shows a computer system 700 that may be used with the examples described herein. The computer system 700 may represent a generic platform that includes components that may be in a server or another computer system. The computer system 700 may be used as a platform for the apparatus 100. The computer system 700 may execute, by a processor (e.g., a single or multiple processors) or other hardware processing circuit, the methods, functions and other processes described herein. These methods, functions and other processes may be embodied as machine readable instructions stored on a computer readable medium, which may be non-transitory, such as hardware storage devices (e.g., RAM (random access memory), ROM (read only memory), EPROM (erasable, programmable ROM), EEPROM (electrically erasable, programmable ROM), hard drives, and flash memory).

[0091] The computer system 700 may include a processor 702 that may implement or execute machine readable instructions performing some or all of the methods, functions and other processes described herein. Commands and data from the processor 702 may be communicated over a communication bus 704. The computer system may also include a main memory 706, such as a random access memory (RAM), where the machine readable instructions and data for the processor 702 may reside during runtime, and a secondary data storage 708, which may be non-volatile and stores machine readable instructions and data. The memory and data storage are examples of computer readable mediums. The memory 706 may include a backend analysis and transformation module 720 including machine readable instructions residing in the memory 706 during runtime and executed by the processor 702. The backend analysis and transformation module 720 may include the modules of the apparatus 100 shown in Figure 1 . [0092] The computer system 700 may include an I/O device 710, such as a keyboard, a mouse, a display, etc. The computer system may include a network interface 712 for connecting to a network. Other known electronic components may be added or substituted in the computer system.

[0093] What has been described and illustrated herein is an example along with some of its variations. The terms, descriptions and figures used herein are set forth by way of illustration only and are not meant as limitations. Many variations are possible within the spirit and scope of the subject matter, which is intended to be defined by the following claims - and their equivalents - in which all terms are meant in their broadest reasonable sense unless otherwise indicated.

Claims

What is claimed is:
1 . A method for backend analysis and transformation, the method comprising: receiving input data that includes a format that is specific to an input source; selecting a transformation from a plurality of available transformations to transform the input data to an output including a format that is specific to an output source; and transforming, by a processor, the input data to generate the output including the format that is specific to the output source based on the selected
transformation, wherein the selected transformation uses: a definition archive that is selected from a plurality of available definition archives to drive the selected transformation of the input data to generate the output including the format that is specific to the output source, wherein the selected definition archive includes a set of instructions that are hot-pluggable at runtime to dynamically allow new solutions related to the selected transformation to be implemented; and external packages that are selected from a plurality of available external packages to implement the selected definition archive, wherein the selected external packages are hot-pluggable at runtime to implement the selected definition archive, and the plurality of available external packages include library bundles to implement different ones of the plurality of available definition archives.
2. The method of claim 1 , wherein transforming the input data to generate the output including the format that is specific to the output source based on the selected transformation further comprises: performing the selected transformation in a Java virtual machine (JVM) execution environment.
3. The method of claim 1 , wherein transforming the input data to generate the output including the format that is specific to the output source based on the selected transformation further comprises: controlling serial and parallel execution of machine readable instructions related to the selected transformation.
4. The method of claim 1 , wherein the selected definition archive includes compiled code, third party Java libraries, and predefined reusable syntax wirings to implement the selected transformation.
5. The method of claim 1 , wherein transforming the input data to generate the output including the format that is specific to the output source based on the selected transformation further comprises: performing the selected transformation independently of an execution framework for the selected transformation.
6. The method of claim 1 , wherein transforming the input data to generate the output including the format that is specific to the output source based on the selected transformation further comprises: using generic data loaders to load the input data, wherein the generic data loaders include a class structure loader that is an object relational mapping (ORM) loader to create a database table structure from a list of classes related to the input data to implement the selected transformation.
7. The method of claim 1 , wherein the selected definition archive includes a model definition Extensible Markup Language (XML) file that defines a complete transformation flow for the selected transformation.
8. The method of claim 1 , wherein the selected external packages include Web Archive (WAR), Enterprise Archive (EAR), and Java Archive (JAR) files to implement the selected definition archive.
9. A backend analysis and transformation apparatus comprising: a processor; and a memory storing machine readable instructions that when executed by the processor cause the processor to: receive input data that includes a format that is specific to an input source; select a transformation from a plurality of available transformations to transform the input data to an output including a format that is specific to an output source; and transform the input data to generate the output including the format that is specific to the output source based on the selected transformation, wherein the selected transformation uses: a definition archive that is selected from a plurality of available definition archives to drive the selected transformation of the input data to generate the output, wherein the selected definition archive includes a set of instructions that are hot-pluggable at runtime to dynamically allow new solutions related to the selected transformation to be implemented, and a model definition file that defines a complete transformation for the selected transformation to implement an execution environment and data flow control that is unique to the selected transformation; and external packages that are selected from a plurality of available external packages to implement the selected definition archive, wherein the selected external packages are hot-pluggable at runtime to implement the selected definition archive, and the plurality of available external packages include library bundles to implement different ones of the plurality of available definition archives.
10. The backend analysis and transformation apparatus according to claim 9, wherein the selected definition archive includes wiring instructions for Java beans wiring and conditional beans wiring at different phases of the selected
transformation.
11 . The backend analysis and transformation apparatus according to claim 9, wherein the machine readable instructions that when executed by the processor further cause the processor to: persist into a Structured Query Language (SQL) query-able form any interlinked Plain Old Java Object (POJO) structures into an internal relational database while preserving data relation links to implement the selected
transformation.
12. The backend analysis and transformation apparatus according to claim 9, wherein the machine readable instructions to transform the input data to generate the output including the format that is specific to the output source based on the selected transformation, further cause the processor to: perform the selected transformation in a Java virtual machine (JVM) execution environment.
13. A non-transitory computer readable medium having stored thereon machine readable instructions to provide backend analysis and transformation, the machine readable instructions, when executed, cause a processor to: receive input data that includes a format that is specific to an input source; select a transformation from a plurality of available transformations to transform the input data to an output including a format that is specific to an output source; and transform the input data to generate the output including the format that is specific to the output source based on the selected transformation, wherein the selected transformation uses a definition archive that is selected from a plurality of available definition archives to drive the selected transformation of the input data to generate the output including the format that is specific to the output source, wherein the selected definition archive includes a set of instructions that are hot- pluggable at runtime to dynamically allow new solutions related to the selected transformation to be implemented.
14. The non-transitory computer readable medium of claim 13, wherein the machine readable instructions to transform the input data to generate the output including the format that is specific to the output source based on the selected transformation, further cause the processor to: use external packages that are selected from a plurality of available external packages to implement the selected definition archive, wherein the selected external packages are hot-pluggable at runtime to implement the selected definition archive, and the plurality of available external packages include library bundles to implement different ones of the plurality of available definition archives.
15. The non-transitory computer readable medium of claim 13, wherein the selected definition archive includes compiled code, third party Java libraries, and predefined reusable syntax wirings to implement the selected transformation.
PCT/US2014/036945 2014-05-06 2014-05-06 Backend analysis and transformation WO2015171117A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/US2014/036945 WO2015171117A1 (en) 2014-05-06 2014-05-06 Backend analysis and transformation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2014/036945 WO2015171117A1 (en) 2014-05-06 2014-05-06 Backend analysis and transformation

Publications (1)

Publication Number Publication Date
WO2015171117A1 true WO2015171117A1 (en) 2015-11-12

Family

ID=54392787

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2014/036945 WO2015171117A1 (en) 2014-05-06 2014-05-06 Backend analysis and transformation

Country Status (1)

Country Link
WO (1) WO2015171117A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060059115A1 (en) * 2004-09-16 2006-03-16 Gutfleisch Michael R System and method for transforming information between data formats
US20070203926A1 (en) * 2006-02-24 2007-08-30 Microsoft Corporation Scalable transformation and configuration of EDI interchanges
US20070271562A1 (en) * 2003-11-03 2007-11-22 Schumacher Larry L Execution environment for data transformation applications
US20090265682A1 (en) * 2008-04-16 2009-10-22 International Business Machines Corporation Method for transforming data from a model into a secondary model to optimize code generation
US20120109988A1 (en) * 2010-10-28 2012-05-03 Honeywell International Inc. System and method for data mapping and information sharing

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070271562A1 (en) * 2003-11-03 2007-11-22 Schumacher Larry L Execution environment for data transformation applications
US20060059115A1 (en) * 2004-09-16 2006-03-16 Gutfleisch Michael R System and method for transforming information between data formats
US20070203926A1 (en) * 2006-02-24 2007-08-30 Microsoft Corporation Scalable transformation and configuration of EDI interchanges
US20090265682A1 (en) * 2008-04-16 2009-10-22 International Business Machines Corporation Method for transforming data from a model into a secondary model to optimize code generation
US20120109988A1 (en) * 2010-10-28 2012-05-03 Honeywell International Inc. System and method for data mapping and information sharing

Similar Documents

Publication Publication Date Title
US8307331B2 (en) User interface, operating system and architecture
US7650590B2 (en) Flexible code generation
Johnson et al. Professional Java development with the Spring framework
US7814404B2 (en) System and method for applying workflow of generic services to component based applications for devices
Abiteboul et al. The Active XML project: an overview
US20060095274A1 (en) Execution engine for business processes
US8006224B2 (en) System and method for unified visualization of two-tiered applications
US20030182625A1 (en) Language and object model for describing MIDlets
US7954107B2 (en) Method and system for integrating the existing web-based system
US7305659B2 (en) Handling parameters in test scripts for computer program applications
US7574692B2 (en) Method for building component-software for execution in a standards-compliant programming environment
US20070067418A1 (en) Object oriented web application framework
US7124401B2 (en) Testing versions of applications
US20030181196A1 (en) Extensible framework for code generation from XML tags
US6772083B2 (en) Computer program test configurations with data containers and test scripts
US20030182626A1 (en) On-demand creation of MIDlets
US8949788B2 (en) Building and packaging software
US9161156B2 (en) Tiles in a mobile application framework
Jendrock et al. The java ee 7 tutorial
US20080147698A1 (en) Data schemata in programming language contracts
Juric et al. WSDL and UDDI extensions for version support in web services
US20070214408A1 (en) Declarative web application for search and retrieval
EP1872204A4 (en) System and method for flexible visual representation of presentation components
WO2006043012A1 (en) Data processing system and method
US8438541B2 (en) Software change management extension for uniformly handling artifacts with relaxed contraints

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14891378

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14891378

Country of ref document: EP

Kind code of ref document: A1