EP3311265B1 - A computing platform and method thereof for searching, executing, and evaluating computational algorithms - Google Patents
A computing platform and method thereof for searching, executing, and evaluating computational algorithms Download PDFInfo
- Publication number
- EP3311265B1 EP3311265B1 EP16812509.4A EP16812509A EP3311265B1 EP 3311265 B1 EP3311265 B1 EP 3311265B1 EP 16812509 A EP16812509 A EP 16812509A EP 3311265 B1 EP3311265 B1 EP 3311265B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- container
- software
- software container
- textual description
- resources
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004422 calculation algorithm Methods 0.000 title claims description 66
- 238000000034 method Methods 0.000 title claims description 27
- 230000006870 function Effects 0.000 claims description 27
- 238000012545 processing Methods 0.000 claims description 15
- 238000009434 installation Methods 0.000 claims description 6
- 239000003795 chemical substances by application Substances 0.000 description 34
- 241001521293 Python Species 0.000 description 19
- 238000004458 analytical method Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 239000008186 active pharmaceutical agent Substances 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000007373 indentation Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/36—Software reuse
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
- G06F11/3428—Benchmarking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45504—Abstract machines for programme code execution, e.g. Java virtual machine [JVM], interpreters, emulators
Definitions
- the disclosure generally relates to the field of computer programming tools, and in particular to a computing platform and method to assist in programming, researching, evaluating, and maintenance of computational algorithms.
- a typical software application interfaces with different modules, e.g., using APIs, and should be programmed to enable its execution on different platforms and operating systems.
- a software application should be available for execution over different types of operating systems, such as Linux®, Mac OS® and Windows®.
- the code should be written such that its execution is supported by any computing environments in the cloud infrastructure.
- resources that include "off-the-shelf" functions, algorithms, and pieces of codes are available.
- Such resources range from academic and scientific publications to code repositories (e.g., GitHub).
- code repositories e.g., GitHub
- such resources are limited as they cannot provide executable code that can be easily evaluated and integrated in a software application or project being researched or developed.
- a scientific publication e.g., in an IEEE® article
- scientific publications do not include any code that can be immediately executed on a computing environment for which the software application is developed. Thus, to evaluate the functionality and performance of the algorithm, executable code should be written and tested on that computing environment. This is a time consuming and error prone task.
- a scientific publication would include a link to code repository that provides a sample code. However, this would still require the programmer to integrate the code in a software program for further evaluation.
- the invention is set forth in independent claims 1, 13, and 14. Certain embodiments disclosed herein includes a method for evaluating computational algorithms. The method comprises receiving a textual description of a computational algorithm; generating a software container based on the received textual description; instantiating a computing environment on a computing device, wherein the computing environment includes computing resources configured to support execution of the software container; executing the software container in the computing environment; and displaying, on a user device, results that are output in response to execution of the software container.
- Certain embodiments disclosed herein also include a system for evaluating computational algorithms.
- the system comprises a processing system; and a memory, the memory containing instructions that, when executed by the processing system, configure the system to: receive a textual description of a computational algorithm; generate a software container based on the received textual description; instantiate a computing environment on a computing device, wherein the computing environment includes computing resources configured to support execution of the software container; execute the software container in the computing environment; and display, on a user device, results that are output in response to execution of the software container.
- Fig. 1 is an example diagram of a networked system 100 utilized to describe the various disclosed embodiments.
- the networked system 100 includes a cloud computing platform 110, a plurality of data repositories 120-1 through 120-N, computing devices 130-1 and 130-2, a server 140, and a data warehouse 150.
- the various elements of the system 100 are communicatively connected to a network 160.
- the network 160 may be, for example, a wide area network (WAN), a local area network (LAN), the Internet, and the like.
- the cloud computing platform 110 may be a private cloud, a public cloud, or a hybrid cloud providing computing resources to applications or services executed therein.
- the plurality of data repositories 120-1 through 120-N (hereinafter individually referred to as a data repository 120 or collectively as data repositories 120) are resources to computational algorithms.
- a computational algorithm is any algorithm that can be utilized in the field of mathematics and/or computer science. Such an algorithm is typically defined as a self-contained step-by-step set of operations to be performed.
- a computational algorithm may be designed to perform calculation, data processing and analysis (such as, image processing, transaction processing), automated reasoning, and more.
- the computational algorithms may be included in text books, scientific or academic publications (e.g., articles or magazines), blogs, manuals, tutorials (textual or video), and so on.
- a computational algorithm saved in a data repository may be described in a format of pseudo code, one or more mathematical equations, a written description, a code sample or samples, or any combination thereof.
- the computing devices 130-1 and 130-2 may include, for example, a personal computer, a laptop, a tablet computer, a smartphone, and the like. Each computing device 130 is utilized by a user to develop a software application, and as such includes an integrated development environment (IDE) 131 or any other tool that allows computer programmers to develop software.
- IDE integrated development environment
- Each computing device 130 further includes an agent 132 locally installed and executed over the computing device 130.
- the agent 132 is configured to scan any code developed in the IDE 131. Based on the code scanning, one or more software containers (or image containers) 133 are generated. The generated containers 133 are sent by the agent 132 to the server 140, to the data warehouse 150, or to both.
- the agent 132 is further configured to interface with the server 140 to retrieve the containers 133 and to integrate the retrieved containers 133 in a software program developed by the user.
- an agent 132 has at least two primary roles: generating the containers 133, and integrating the generated containers 133 in a software program developed in the IDE 131.
- the integration of the containers includes compiling or loading the containers 133 into code provided by the IDE 131.
- the computing device 130-1 is operated by an author of the algorithm (e.g., the agent 132-1 generates containers 133-1), while the computing device 130-2 is used by a user of the containers (e.g., the agent 132-2 may integrate the containers 133-1 in the code).
- any user of the computing devices 130-1 and 130-2 may be, but is not limited to, an engineer, a programmer, a researcher, a developer, or any person with skill in the field of computer programming.
- an author using the computing device 130-1, develops an algorithm to compute an average of student grades.
- the agent 132-1 is configured to scan this algorithm to provide a container descriptor and to generate a container 133-1 respective of the container descriptor.
- the generated container 133-1 encapsulates all the software resources required for executing the algorithm.
- the generated container 133-1 includes a data set serving as an input for the algorithm. The process for providing the container descriptor and generating the containers 133 is discussed in detail below.
- the container descriptor can be generated based on any format that the algorithm is presented in, e.g., a pseudo code, a code sample, an executable code, equations, and so on. It should be further emphasized that the algorithm can be defined using the IDE 131-1, locally saved in the computing device 130-1, or downloaded from any data repositories 120.
- a user of the computing device 130-2 can download the container 133-1 (as in the above example, to compute an average) and integrate the container 133-1 in any software program developed by means of the IDE 131-2.
- the process of the downloading, integrating, or otherwise executing the container 133-1 on the computing device 130-2 is facilitated by means of the agent 132-2.
- the agent 132-2 provides the container 133-1 with a complete set of interfaces to the computing environment. That is, any input to, or output generated by, the container 133-1 is received or sent through the agent 132-2.
- the agent 132-2 interfaces between the container 133-1 to the IDE 131-2, the server 140, and an operating system and file system of the computing device 130-2.
- the agent 132-2 is also configured to install all software resources required for the execution of the container 133-1 on the computing device 130-2.
- the container 133-1 includes such software resources and the agent 132-2 is configured to select those resources that are needed for the proper execution of the container's 133-1 code. It should be noted that the agent 132-2 can check which resources are required for installation and which are not. For example, assuming the container's 133-1 code is in Python, the agent installs a Python native in the container 133-1. As such, the container 133-1 including the Python installation can be executed on any computer regardless of whether the computer includes Python native. Alternatively, the container 133-1 may call to install Python native on-the-fly.
- the agent 132-2 detects that the operating system of the device 130-2 does not support Python, then the agent 132-2 causes installation of the native Python on the device 130-2, on the fly.
- the user is not required to install any files, and these techniques may further allow seamless sharing and execution across different devices.
- the agent 132-2 is further configured to provide the IDE 133-1 with the code included in the container 133-2 to enable the display of such code in the IDE 133-1.
- the server 140 is configured to receive a container descriptor from an agent 132-1 and to generate the container 133-1 respective thereof.
- containers 133 can be executed by the server 140, and their inputs/outputs will be communicated to the respective agent 132 through an API.
- a container 133-1 for computing averages can receive an input data set (e.g., a list of student grades) provided to the server 140 by the agent 132-2, and upon execution of the container 133-1 by the server 140, the average is returned to the agent 132-2.
- the input data set is part of the container 133-1.
- the server 140 is configured to create or instantiate a computing environment that supports the execution of the container 133.
- a container 133 includes a Python code
- a virtual machine (not shown) that can run Python is instantiated by the server 140.
- the creation of computing environments for running containers 133 can be performed on-demand (e.g., upon receiving a request to run a container).
- computing environments utilized by "popular" containers e.g., computing environments utilized above a predefined threshold number of times during a given time period) are always available.
- the resources produced during the execution of a container 133 are cached, for example, in a cache memory (not shown) connected to the server 140.
- the cached resources may include, for example, installation files, compilation files, results output by the container, combinations thereof, and the like.
- the cache memory can be shared among different computing environments (e.g., "workers") created to execute the containers. This sharing of cache memory allows for sharing cached resources among the computing environments. It should be appreciated that the cached resources produced in one worker can be used by other workers without requiring re-compilation, thereby accelerating the execution and accessibility to containers. It should be noted that the cached resources are available to containers executed by the agents 132 on user devices 130 or to the server 140.
- cached resources is based in part on the memory's state during code execution including all variables, matrixes, figures, intermediate/temporarily files, result files.
- the cached resources can be saved in the platform's centralized data repository, on a ready to execute "warm” worker, or both with a synchronizing mechanism.
- caching the resources include sharing the resources between executions of a container by the same or different accounts and users.
- the trigger to use cached resources is when, for example, the state of code, input data, packages, versions and configuration are detected as the same or similar enough to generate the same resources.
- execution of empty spaces or lines, comments, or certain pieces of code can be avoided using the cached resources. That is, such resources can be utilized instead of executing their respective pieces of code. The execution can them resume from that point to either execute new code according to user request or predefined code that will be executed from the shared resources point and forward automatically.
- shared resources can also be utilized during the compilation stage, where certain compilation steps are not performed. To this end, the state of the code and cached (shared) resources are identified, and compilation, execution and/or synchronization of shared resources is not performed.
- the server 140 is further configured to create the containers 133 based on the container descriptors provided by the agents 132.
- the generated containers 133 are saved in the data warehouse 150. The process for generating the container descriptors and containers is described below.
- a search for containers 133 and the respective algorithms they carry out can be performed by any device communicatively connected to the server 140.
- the containers are saved in the data warehouse 150.
- Metadata is generated and associated with each container 133 saved in the data warehouse 150.
- the metadata may be generated by the server 140 and/or by an agent 132.
- the metadata may include, but is not limited to, a container name, a container creation date, an algorithm name, an author of the algorithm, similar or related algorithms, user interaction, combinations thereof, and so on.
- the user interaction metadata lists how users downloaded or executed a specific container, the number of repetitions of a container per user, a benchmark, combinations thereof, and so on.
- a ranking process is implemented to rank matching containers to a user query respective of the generated metadata.
- the search may be performed through a web interface (e.g., a portal) communicatively connected to the server 140.
- the ranking may be based, in part, on the user interaction information contained in the metadata. For example, a ranking score indicative of the popularity of a container may be computed using the number of runs, the number of views, and the number of downloads. The containers and/or their respective printed publications will be ranked based on their computed scores.
- the evaluation of the performance and/or functionality of an algorithm can be also performed through the web interface.
- An exemplary and non-limiting screenshot of a webpage 200 for evaluating an algorithm is provided in Fig. 2 .
- the webpage 200 is rendered by the web interface.
- the algorithm is for a recognition program written in Matlab® code.
- the webpage 200 may display the code associated with a container generated for the algorithm.
- the algorithm is composed from 3 files listed in the file list 210. The contents of each file can be viewed and edited in the section 220 of the webpage 200. Any changes to the code are saved to the data warehouse 150. Also, the changes are associated with the respective container.
- the respective container generated for the recognition algorithm (and any changes therein) is executed by the server 140.
- the results of the execution are displayed in the results section 240.
- the server 140 is configured to benchmark an algorithm through its respective container.
- an input data set is predefined to be utilized by containers designed to execute similar algorithms.
- the server 140 is configured to execute each container and to measure performance parameters.
- the parameters may be global parameters and/or specific parameters.
- the global parameters are agnostic to the purpose of the algorithm, while the specific parameters are related to the purpose/function of the algorithm.
- global parameters may include execution time, memory, CPU and/or bandwidth utilization, and so on.
- the specific parameters may be, for example, a compression ratio for lossless compression algorithms.
- the agent is configured to generate a user interface, e.g., a graphical user interface, based on the container's code.
- a user interface e.g., a graphical user interface
- the developer of the container would not be required to code any GUI for inputting data or outputting the results.
- the developer is merely required to specify which variables are input variables.
- the agent is configured to scan the code, to identify the specified input variables and their types, to generate GUI elements respective of such variables, and to display the generated GUI elements according to a specific order in which the input should be received.
- the agent is further configured to provide an output window for displaying the execution's results.
- the generated GUI enables easy and fast interaction with the executed computational algorithm. For example, a user evaluating the computational algorithm is not required to enter a series of datasets through a command line.
- the number of downloads of containers and/or the execution of the containers by a user are tracked. This allows for monetization of the usage of containers, for example, via compensation per download or per execution.
- each container generated according to the disclosed embodiments can be shared among different users, thereby allowing collaboration among users.
- the sharing of containers can be on many levels, i.e., the entire container, input data sets, results, the container's code, or a combination thereof, can be shared.
- two or more users can work on code of a single container.
- a version (or source) control is implemented to record changes to a container's code so that a user can recall specific versions as needed.
- a global container is created and integrated in the code.
- the global container can be executed on any computing (or programming) environment. When executed, the global container can call other different containers to run the respective algorithm.
- a software license is required to execute a code.
- a Matlab® license is needed to run Matlab® code.
- a license manager is implemented by any of the agents 132 on the server 140. The license manager checks if a valid license exists on the local device before running a container that contains a code that requires a license. If so, execution of the container on the local device is enabled. Otherwise, the request is sent to the server 140 to grant a temporary license for executing the container.
- the temporary license may be restricted to a predefined time period, a number of executions of the same container, and/or a number of executions of the different containers.
- a monetization method is associated with the temporary licenses. That is, a user granting such a license may be billed for the service.
- containers stored in the data warehouse 150 can be associated with computational algorithms included in printed text books, scientific, and/or academic publications. The association is achieved using, for example, a QR code minted in the printed publication.
- a user by means of a device 130, can access the containers in the data warehouse 150 by simply scanning the QR code.
- Containers accessed through QR codes are retrieved and saved in the server 140 to allow execution by the user device 130.
- all such containers are also listed in a user account, allowing the user to easily access and execute the containers at a later time.
- the server 140 typically includes a processing system (not shown) connected to a memory (not shown).
- the memory contains a plurality of instructions that are executed by the processing system.
- the memory may include machine-readable media for storing software.
- Software shall be construed broadly to mean any type of instructions, whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise. Instructions may include code (e.g., in source code format, binary code format, executable code format, or any other suitable format of code). The instructions, when executed by the one or more processors, cause the processing system to perform the various functions described herein.
- the processing system may comprise or be a component of a larger processing system implemented with one or more processors.
- the one or more processors may be implemented with any combination of general-purpose microprocessors, microcontrollers, digital signal processors (DSPs), field programmable gate array (FPGAs), graphics processing units (GPUs), programmable logic devices (PLDs), controllers, state machines, gated logic, discrete hardware components, dedicated hardware finite state machines, or any other suitable entities that can perform calculations or other manipulations of information.
- DSPs digital signal processors
- FPGAs field programmable gate array
- GPUs graphics processing units
- PLDs programmable logic devices
- controllers state machines, gated logic, discrete hardware components, dedicated hardware finite state machines, or any other suitable entities that can perform calculations or other manipulations of information.
- the agents 132-1 and 132-2 can operate and be implemented as a stand-alone program or, alternatively, can communicate and be integrated with other programs or applications executed in the client device 130.
- either or each of the agents 132-1 and 132-2 can be realized as a hardware component, such as a general-purpose microprocessor, microcontroller, a DSP, a FPGA, a PLD, a controller, a state machine, gated logic, a discrete hardware component, a dedicated hardware finite state machine, or any other suitable entities that can perform calculations or other manipulations of information.
- a hardware component such as a general-purpose microprocessor, microcontroller, a DSP, a FPGA, a PLD, a controller, a state machine, gated logic, a discrete hardware component, a dedicated hardware finite state machine, or any other suitable entities that can perform calculations or other manipulations of information.
- the server 140 may reside in the cloud computing platform 110, a different cloud computing platform or datacenter. Moreover, in an embodiment, there may be a plurality of servers 140 operating as described hereinabove and configured to either have one as a standby, to share the load between them, or to split the functions between them.
- Fig. 3 depicts an example flowchart 300 illustrating a method for generating a container for a computational algorithm according to one embodiment.
- a textual description of a computational algorithm is received.
- the textual description may be in a format of a pseudo code, one or more mathematical equations, a written description, a code sample or samples, any combination thereof, and the like.
- the programming language of the received algorithm is identified.
- the identification may be based on the file extension name, reserved words in the received textual description, keywords, syntax characters (e.g., ' ⁇ ' ' ⁇ '; indentations, etc.), and so on.
- the programming language is any high-level programming language that may or may not require compiling by another program.
- the programming language is a scripting language.
- the received algorithm's textual description is analyzed to generate a dependency tree.
- a minimal set of functions that should be included in the container is determined.
- the output of S320 is utilized for the analysis at S340. The operation of S330 and S340 will now be described with reference to Fig. 4 that depicts a schematic diagram of a dependency tree 400.
- the textual description (code) of the algorithm utilized to generate the example dependency tree 400 is:
- the textual description is a Python script.
- execution of a Python script requires installation of a Python environment.
- any Python environment typically requires native Python.
- the received code is scanned to determine which libraries and their functions (if any) are needed for the code's execution. Such libraries and functions are identified based on the "from ⁇ > import ⁇ >" syntax.
- the required libraries are: 'path' 411 from 'sys' 410 library, 'display' 421 and 'image' 422 from the 'wand' 420 library, 'rmtree' 431 from the 'shutil' 430 library, as well ' os ' 440.
- the required functions are ' display ' 421-1 from the 'display' library, ' get_resized_image ' 422-1 from the 'image' library, and ' chdir ' 441 from the ' os ' library.
- the analysis of the code indicates that the libraries ' sys ' 410 and ' os ' 440 are part of the native Python. Therefore, the minimal set of functions (in addition to native Python) that should be installed (or included) in the container are the 'display' 422-1 and ' get_resized_image ' 423-1.
- the ' chdir ' is part of the native Python that must be included in a container for executing the above example code.
- the analysis of the code and the generation of the dependency tree ensure generation of compact containers. That is, for example, the 'wand' library 420 includes many sub-libraries, and each sub-library includes many functions. Thus, instead of generating a container that contains the entire 'wand' library 420, only the ' get_resized_image ' 422-1 would be included in the container.
- a container descriptor is created based on the analysis performed at S340.
- the container descriptor lists, at least, a minimal set of resources and their dependencies that should be included in the container. Referring to the example described herein above with respect to Fig. 4 , the container descriptor would list a Python native and the functions ' display ' and ' get_resized_image '.
- the container descriptor is created in a format that is supported by the type of container to be generated. For example, if such container type is Docker, the container descriptor format is a Dockerfile.
- a Dockerfile (container descriptor) is a script composed of various instructions and arguments that, when successively executed, perform actions on a container descriptor in order to generate a new container.
- a container for executing the received algorithm is generated using the container descriptor.
- such container may be a Docker container.
- the container is generated from a "base" container descriptor which defines mandatory resources for the proper execution of the algorithm.
- the base container descriptor may define native Python and the new container would include the additional functions ' display ' and ' get_resized_image '.
- the user can edit or modify the container descriptor or any container generated thereof.
- the various embodiments have been discussed herein with a reference to a software container.
- the software container can be replaced with a virtual machine, a light virtual machine, or any applicable virtualization technologies without departing from the scope of the disclosure.
- the various embodiments disclosed herein can be implemented as hardware, firmware, software, or any combination thereof.
- the software is preferably implemented as an application program tangibly embodied on a program storage unit or computer readable medium consisting of parts, or of certain devices and/or a combination of devices.
- the application program may be uploaded to, and executed by, a machine comprising any suitable architecture.
- the machine is implemented on a computer platform having hardware such as one or more central processing units (“CPUs"), a memory, and input/output interfaces.
- CPUs central processing units
- the computer platform may also include an operating system and microinstruction code.
- a non-transitory computer readable medium is any computer readable medium except for a transitory propagating signal.
- any reference to an element herein using a designation such as “first,” “second,” and so forth does not generally limit the quantity or order of those elements. Rather, these designations are generally used herein as a convenient method of distinguishing between two or more elements or instances of an element. Thus, a reference to first and second elements does not mean that only two elements may be employed there or that the first element must precede the second element in some manner. Also, unless stated otherwise a set of elements comprises one or more elements.
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Stored Programmes (AREA)
Description
- The disclosure generally relates to the field of computer programming tools, and in particular to a computing platform and method to assist in programming, researching, evaluating, and maintenance of computational algorithms.
- As the development of software applications evolves, programming accurate and efficient code has become a real challenge. A typical software application interfaces with different modules, e.g., using APIs, and should be programmed to enable its execution on different platforms and operating systems. For example, a software application (app) should be available for execution over different types of operating systems, such as Linux®, Mac OS® and Windows®. Furthermore, as execution of most software applications have been migrated to cloud computing infrastructures, the code should be written such that its execution is supported by any computing environments in the cloud infrastructure. Thus, in sum, the complexity of software applications imposes a real challenge on programmers.
- In order to enable the development of rapid creation, setup, and deployment of applications, resources that include "off-the-shelf" functions, algorithms, and pieces of codes are available. Such resources range from academic and scientific publications to code repositories (e.g., GitHub). However, such resources are limited as they cannot provide executable code that can be easily evaluated and integrated in a software application or project being researched or developed. For example, a scientific publication (e.g., in an IEEE® article) would include a textual description of an algorithm or at best its pseudo code.
- In most cases, scientific publications do not include any code that can be immediately executed on a computing environment for which the software application is developed. Thus, to evaluate the functionality and performance of the algorithm, executable code should be written and tested on that computing environment. This is a time consuming and error prone task. At best, a scientific publication would include a link to code repository that provides a sample code. However, this would still require the programmer to integrate the code in a software program for further evaluation.
- Furthermore, due to the voluminous amount of academic and scientific publications, open-source codes, and other algorithm resources, it is almost impossible to effectively search for the right algorithm that would fit into a software application and would perform an intended function.
- For example, in Frank F. Tsui ET AL: "Essentials of software engineering, third edition", 1 January 2014, Burlington, Mass, ISBN: 978-1-4496-9199-8, an arrangement for building software using compiling is disclosed.
- It would therefore be advantageous to provide a solution that overcomes the deficiencies noted above.
- A summary of several example embodiments of the disclosure follows. This summary is provided for the convenience of the reader to provide a basic understanding of such embodiments and does not wholly define the breadth of the disclosure. This summary is not an extensive overview of all contemplated embodiments, and is intended to neither identify key or critical elements of all embodiments nor to delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more embodiments in a simplified form as a prelude to the more detailed description that is presented later. For convenience, the term "some embodiments" may be used herein to refer to a single embodiment or multiple embodiments of the disclosure.
- The invention is set forth in
independent claims 1, 13, and 14. Certain embodiments disclosed herein includes a method for evaluating computational algorithms. The method comprises receiving a textual description of a computational algorithm; generating a software container based on the received textual description; instantiating a computing environment on a computing device, wherein the computing environment includes computing resources configured to support execution of the software container; executing the software container in the computing environment; and displaying, on a user device, results that are output in response to execution of the software container. - Certain embodiments disclosed herein also include a system for evaluating computational algorithms. The system comprises a processing system; and a memory, the memory containing instructions that, when executed by the processing system, configure the system to: receive a textual description of a computational algorithm; generate a software container based on the received textual description; instantiate a computing environment on a computing device, wherein the computing environment includes computing resources configured to support execution of the software container; execute the software container in the computing environment; and display, on a user device, results that are output in response to execution of the software container.
- The subject matter disclosed herein is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other objects, features and advantages of the disclosed embodiments will be apparent from the following detailed description taken in conjunction with the accompanying drawings.
-
Figure 1 is a schematic diagram of a networked system utilized to describe the disclosed embodiments. -
Figure 2 is an exemplary and non-limiting screenshot of a webpage rendered for evaluating an algorithm according to one embodiment. -
Figure 3 is a flowchart illustrating a method for generating a container according to one embodiment. -
Figure 4 is a schematic diagram of a dependency tree generated according to one embodiment. - It is important to note that the embodiments disclosed herein are only examples of the many advantageous uses of the innovative teachings herein. In general, statements made in the specification of the present application do not necessarily limit any of the various claims. Moreover, some statements may apply to some inventive features but not to others. In general, unless otherwise indicated, singular elements may be in plural and vice versa with no loss of generality. In the drawings, like numerals refer to like parts through several views.
-
Fig. 1 is an example diagram of a networkedsystem 100 utilized to describe the various disclosed embodiments. Thenetworked system 100 includes acloud computing platform 110, a plurality of data repositories 120-1 through 120-N, computing devices 130-1 and 130-2, aserver 140, and adata warehouse 150. The various elements of thesystem 100 are communicatively connected to anetwork 160. Thenetwork 160 may be, for example, a wide area network (WAN), a local area network (LAN), the Internet, and the like. - The
cloud computing platform 110 may be a private cloud, a public cloud, or a hybrid cloud providing computing resources to applications or services executed therein. The plurality of data repositories 120-1 through 120-N (hereinafter individually referred to as adata repository 120 or collectively as data repositories 120) are resources to computational algorithms. In the context of the disclosed embodiments, a computational algorithm is any algorithm that can be utilized in the field of mathematics and/or computer science. Such an algorithm is typically defined as a self-contained step-by-step set of operations to be performed. As an example, a computational algorithm may be designed to perform calculation, data processing and analysis (such as, image processing, transaction processing), automated reasoning, and more. - The computational algorithms may be included in text books, scientific or academic publications (e.g., articles or magazines), blogs, manuals, tutorials (textual or video), and so on. A computational algorithm saved in a data repository may be described in a format of pseudo code, one or more mathematical equations, a written description, a code sample or samples, or any combination thereof.
- The computing devices 130-1 and 130-2 may include, for example, a personal computer, a laptop, a tablet computer, a smartphone, and the like. Each computing device 130 is utilized by a user to develop a software application, and as such includes an integrated development environment (IDE) 131 or any other tool that allows computer programmers to develop software.
- Each computing device 130 further includes an agent 132 locally installed and executed over the computing device 130. According to various embodiments, the agent 132 is configured to scan any code developed in the IDE 131. Based on the code scanning, one or more software containers (or image containers) 133 are generated. The generated containers 133 are sent by the agent 132 to the
server 140, to thedata warehouse 150, or to both. The agent 132 is further configured to interface with theserver 140 to retrieve the containers 133 and to integrate the retrieved containers 133 in a software program developed by the user. - Thus, according to the disclosed embodiments, an agent 132 has at least two primary roles: generating the containers 133, and integrating the generated containers 133 in a software program developed in the IDE 131. In an embodiment, the integration of the containers includes compiling or loading the containers 133 into code provided by the IDE 131.
- These roles are realized by the agents 132 operable in the computing devices 130-1 and 130-2. In an example embodiment, the computing device 130-1 is operated by an author of the algorithm (e.g., the agent 132-1 generates containers 133-1), while the computing device 130-2 is used by a user of the containers (e.g., the agent 132-2 may integrate the containers 133-1 in the code).
- It should be noted that the author and/or any user of the computing devices 130-1 and 130-2 may be, but is not limited to, an engineer, a programmer, a researcher, a developer, or any person with skill in the field of computer programming.
- As an example, an author, using the computing device 130-1, develops an algorithm to compute an average of student grades. The agent 132-1 is configured to scan this algorithm to provide a container descriptor and to generate a container 133-1 respective of the container descriptor. The generated container 133-1 encapsulates all the software resources required for executing the algorithm. In an embodiment, the generated container 133-1 includes a data set serving as an input for the algorithm. The process for providing the container descriptor and generating the containers 133 is discussed in detail below.
- It should be emphasized that the container descriptor can be generated based on any format that the algorithm is presented in, e.g., a pseudo code, a code sample, an executable code, equations, and so on. It should be further emphasized that the algorithm can be defined using the IDE 131-1, locally saved in the computing device 130-1, or downloaded from any
data repositories 120. - A user of the computing device 130-2 can download the container 133-1 (as in the above example, to compute an average) and integrate the container 133-1 in any software program developed by means of the IDE 131-2. The process of the downloading, integrating, or otherwise executing the container 133-1 on the computing device 130-2 is facilitated by means of the agent 132-2.
- Specifically, the agent 132-2 provides the container 133-1 with a complete set of interfaces to the computing environment. That is, any input to, or output generated by, the container 133-1 is received or sent through the agent 132-2. As an example, the agent 132-2 interfaces between the container 133-1 to the IDE 131-2, the
server 140, and an operating system and file system of the computing device 130-2. The agent 132-2 is also configured to install all software resources required for the execution of the container 133-1 on the computing device 130-2. - According to the disclosed embodiments, the container 133-1 includes such software resources and the agent 132-2 is configured to select those resources that are needed for the proper execution of the container's 133-1 code. It should be noted that the agent 132-2 can check which resources are required for installation and which are not. For example, assuming the container's 133-1 code is in Python, the agent installs a Python native in the container 133-1. As such, the container 133-1 including the Python installation can be executed on any computer regardless of whether the computer includes Python native. Alternatively, the container 133-1 may call to install Python native on-the-fly. Thus, for example, if the agent 132-2 detects that the operating system of the device 130-2 does not support Python, then the agent 132-2 causes installation of the native Python on the device 130-2, on the fly. Thus, in some embodiments, the user is not required to install any files, and these techniques may further allow seamless sharing and execution across different devices.
- In another embodiment, the agent 132-2 is further configured to provide the IDE 133-1 with the code included in the container 133-2 to enable the display of such code in the IDE 133-1.
- Certain tasks performed by the agents 132 can be carried out, alternatively or collectively, by the
server 140. Specifically, in an embodiment, theserver 140 is configured to receive a container descriptor from an agent 132-1 and to generate the container 133-1 respective thereof. In yet another embodiment, containers 133 can be executed by theserver 140, and their inputs/outputs will be communicated to the respective agent 132 through an API. As an example, a container 133-1 for computing averages can receive an input data set (e.g., a list of student grades) provided to theserver 140 by the agent 132-2, and upon execution of the container 133-1 by theserver 140, the average is returned to the agent 132-2. The input data set is part of the container 133-1. - In order to allow execution of a container 133, the
server 140 is configured to create or instantiate a computing environment that supports the execution of the container 133. For example, if a container 133 includes a Python code, a virtual machine (not shown) that can run Python is instantiated by theserver 140. In an embodiment, the creation of computing environments for running containers 133 can be performed on-demand (e.g., upon receiving a request to run a container). In another embodiment, computing environments utilized by "popular" containers (e.g., computing environments utilized above a predefined threshold number of times during a given time period) are always available. - In a further embodiment, the resources produced during the execution of a container 133 are cached, for example, in a cache memory (not shown) connected to the
server 140. The cached resources may include, for example, installation files, compilation files, results output by the container, combinations thereof, and the like. In another embodiment, the cache memory can be shared among different computing environments (e.g., "workers") created to execute the containers. This sharing of cache memory allows for sharing cached resources among the computing environments. It should be appreciated that the cached resources produced in one worker can be used by other workers without requiring re-compilation, thereby accelerating the execution and accessibility to containers. It should be noted that the cached resources are available to containers executed by the agents 132 on user devices 130 or to theserver 140. - Specifically, cached resources is based in part on the memory's state during code execution including all variables, matrixes, figures, intermediate/temporarily files, result files. The cached resources can be saved in the platform's centralized data repository, on a ready to execute "warm" worker, or both with a synchronizing mechanism. Thus, caching the resources include sharing the resources between executions of a container by the same or different accounts and users.
- The trigger to use cached resources is when, for example, the state of code, input data, packages, versions and configuration are detected as the same or similar enough to generate the same resources.
- As an example, execution of empty spaces or lines, comments, or certain pieces of code can be avoided using the cached resources. That is, such resources can be utilized instead of executing their respective pieces of code. The execution can them resume from that point to either execute new code according to user request or predefined code that will be executed from the shared resources point and forward automatically.
- In an embodiment, shared resources can also be utilized during the compilation stage, where certain compilation steps are not performed. To this end, the state of the code and cached (shared) resources are identified, and compilation, execution and/or synchronization of shared resources is not performed.
- In another embodiment, the
server 140 is further configured to create the containers 133 based on the container descriptors provided by the agents 132. The generated containers 133 are saved in thedata warehouse 150. The process for generating the container descriptors and containers is described below. - According to some embodiments, a search for containers 133 and the respective algorithms they carry out can be performed by any device communicatively connected to the
server 140. As noted above, the containers are saved in thedata warehouse 150. - In order to create searchable containers 133, metadata is generated and associated with each container 133 saved in the
data warehouse 150. The metadata may be generated by theserver 140 and/or by an agent 132. The metadata may include, but is not limited to, a container name, a container creation date, an algorithm name, an author of the algorithm, similar or related algorithms, user interaction, combinations thereof, and so on. The user interaction metadata lists how users downloaded or executed a specific container, the number of repetitions of a container per user, a benchmark, combinations thereof, and so on. In an exemplary embodiment, a ranking process is implemented to rank matching containers to a user query respective of the generated metadata. The search may be performed through a web interface (e.g., a portal) communicatively connected to theserver 140. - In an embodiment, the ranking may be based, in part, on the user interaction information contained in the metadata. For example, a ranking score indicative of the popularity of a container may be computed using the number of runs, the number of views, and the number of downloads. The containers and/or their respective printed publications will be ranked based on their computed scores.
- According to some disclosed embodiments, the evaluation of the performance and/or functionality of an algorithm can be also performed through the web interface. An exemplary and non-limiting screenshot of a
webpage 200 for evaluating an algorithm is provided inFig. 2 . Thewebpage 200 is rendered by the web interface. - In the example shown in
Fig. 2 , the algorithm is for a recognition program written in Matlab® code. Thewebpage 200 may display the code associated with a container generated for the algorithm. In this example, the algorithm is composed from 3 files listed in thefile list 210. The contents of each file can be viewed and edited in thesection 220 of thewebpage 200. Any changes to the code are saved to thedata warehouse 150. Also, the changes are associated with the respective container. - Once the user clicks the "Run"
button 230, the respective container generated for the recognition algorithm (and any changes therein) is executed by theserver 140. The results of the execution are displayed in theresults section 240. - According to various embodiments, the
server 140 is configured to benchmark an algorithm through its respective container. To this end, an input data set is predefined to be utilized by containers designed to execute similar algorithms. Theserver 140 is configured to execute each container and to measure performance parameters. The parameters may be global parameters and/or specific parameters. The global parameters are agnostic to the purpose of the algorithm, while the specific parameters are related to the purpose/function of the algorithm. For example, global parameters may include execution time, memory, CPU and/or bandwidth utilization, and so on. The specific parameters may be, for example, a compression ratio for lossless compression algorithms. - In a further embodiment, the agent is configured to generate a user interface, e.g., a graphical user interface, based on the container's code. The developer of the container would not be required to code any GUI for inputting data or outputting the results. The developer is merely required to specify which variables are input variables.
- The agent is configured to scan the code, to identify the specified input variables and their types, to generate GUI elements respective of such variables, and to display the generated GUI elements according to a specific order in which the input should be received. The agent is further configured to provide an output window for displaying the execution's results.
- The generated GUI enables easy and fast interaction with the executed computational algorithm. For example, a user evaluating the computational algorithm is not required to enter a series of datasets through a command line.
- In an embodiment, the number of downloads of containers and/or the execution of the containers by a user are tracked. This allows for monetization of the usage of containers, for example, via compensation per download or per execution.
- As noted above, each container generated according to the disclosed embodiments can be shared among different users, thereby allowing collaboration among users. The sharing of containers can be on many levels, i.e., the entire container, input data sets, results, the container's code, or a combination thereof, can be shared. In an embodiment, two or more users can work on code of a single container. To this end, a version (or source) control is implemented to record changes to a container's code so that a user can recall specific versions as needed.
- It should be noted that the disclosed embodiments allow a programmer to develop a software application including a plurality of containers that may execute the same or different algorithms. Specifically, in an embodiment, a global container is created and integrated in the code. The global container can be executed on any computing (or programming) environment. When executed, the global container can call other different containers to run the respective algorithm.
- Typically, a software license is required to execute a code. For example, a Matlab® license is needed to run Matlab® code. According to various disclosed embodiments, a license manager is implemented by any of the agents 132 on the
server 140. The license manager checks if a valid license exists on the local device before running a container that contains a code that requires a license. If so, execution of the container on the local device is enabled. Otherwise, the request is sent to theserver 140 to grant a temporary license for executing the container. The temporary license may be restricted to a predefined time period, a number of executions of the same container, and/or a number of executions of the different containers. In an embodiment, a monetization method is associated with the temporary licenses. That is, a user granting such a license may be billed for the service. - In another embodiment, containers stored in the
data warehouse 150 can be associated with computational algorithms included in printed text books, scientific, and/or academic publications. The association is achieved using, for example, a QR code minted in the printed publication. A user, by means of a device 130, can access the containers in thedata warehouse 150 by simply scanning the QR code. Containers accessed through QR codes are retrieved and saved in theserver 140 to allow execution by the user device 130. In an embodiment, all such containers are also listed in a user account, allowing the user to easily access and execute the containers at a later time. - In some implementations, the
server 140 typically includes a processing system (not shown) connected to a memory (not shown). The memory contains a plurality of instructions that are executed by the processing system. Specifically, the memory may include machine-readable media for storing software. Software shall be construed broadly to mean any type of instructions, whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise. Instructions may include code (e.g., in source code format, binary code format, executable code format, or any other suitable format of code). The instructions, when executed by the one or more processors, cause the processing system to perform the various functions described herein. - The processing system may comprise or be a component of a larger processing system implemented with one or more processors. The one or more processors may be implemented with any combination of general-purpose microprocessors, microcontrollers, digital signal processors (DSPs), field programmable gate array (FPGAs), graphics processing units (GPUs), programmable logic devices (PLDs), controllers, state machines, gated logic, discrete hardware components, dedicated hardware finite state machines, or any other suitable entities that can perform calculations or other manipulations of information.
- In some implementations, the agents 132-1 and 132-2 can operate and be implemented as a stand-alone program or, alternatively, can communicate and be integrated with other programs or applications executed in the client device 130.
- In certain configurations, either or each of the agents 132-1 and 132-2 can be realized as a hardware component, such as a general-purpose microprocessor, microcontroller, a DSP, a FPGA, a PLD, a controller, a state machine, gated logic, a discrete hardware component, a dedicated hardware finite state machine, or any other suitable entities that can perform calculations or other manipulations of information.
- It should be understood that the embodiments disclosed herein are not limited to the specific architecture illustrated in
Fig. 1 and other architectures may be equally used without departing from the scope of the disclosed embodiments. Specifically, theserver 140 may reside in thecloud computing platform 110, a different cloud computing platform or datacenter. Moreover, in an embodiment, there may be a plurality ofservers 140 operating as described hereinabove and configured to either have one as a standby, to share the load between them, or to split the functions between them. -
Fig. 3 depicts anexample flowchart 300 illustrating a method for generating a container for a computational algorithm according to one embodiment. - At S310, a textual description of a computational algorithm is received. The textual description may be in a format of a pseudo code, one or more mathematical equations, a written description, a code sample or samples, any combination thereof, and the like.
- At S320, if available, the programming language of the received algorithm is identified. The identification may be based on the file extension name, reserved words in the received textual description, keywords, syntax characters (e.g., '{' '}'; indentations, etc.), and so on. The programming language is any high-level programming language that may or may not require compiling by another program. Preferably, the programming language is a scripting language.
- At S330, the received algorithm's textual description is analyzed to generate a dependency tree. At S340, based, in part, on the dependency tree, a minimal set of functions that should be included in the container is determined. In an embodiment, the output of S320 is utilized for the analysis at S340. The operation of S330 and S340 will now be described with reference to
Fig. 4 that depicts a schematic diagram of adependency tree 400. -
- The textual description is a Python script. Typically, execution of a Python script requires installation of a Python environment. Further, any Python environment typically requires native Python. The received code is scanned to determine which libraries and their functions (if any) are needed for the code's execution. Such libraries and functions are identified based on the "from <> import <>" syntax.
- As shown in
Fig. 4 , the required libraries are: 'path' 411 from 'sys' 410 library, 'display' 421 and 'image' 422 from the 'wand' 420 library, 'rmtree' 431 from the 'shutil' 430 library, as well 'os' 440. The required functions are 'display' 421-1 from the 'display' library, 'get_resized_image' 422-1 from the 'image' library, and 'chdir' 441 from the 'os' library. - The analysis of the code (using, e.g., the identified programming language) indicates that the libraries 'sys' 410 and 'os' 440 are part of the native Python. Therefore, the minimal set of functions (in addition to native Python) that should be installed (or included) in the container are the 'display' 422-1 and 'get_resized_image' 423-1. The 'chdir' is part of the native Python that must be included in a container for executing the above example code.
- It should be noted that the analysis of the code and the generation of the dependency tree ensure generation of compact containers. That is, for example, the 'wand'
library 420 includes many sub-libraries, and each sub-library includes many functions. Thus, instead of generating a container that contains the entire 'wand'library 420, only the 'get_resized_image' 422-1 would be included in the container. - Referring back to
Fig. 3 , where at S350, a container descriptor is created based on the analysis performed at S340. The container descriptor lists, at least, a minimal set of resources and their dependencies that should be included in the container. Referring to the example described herein above with respect toFig. 4 , the container descriptor would list a Python native and the functions 'display' and 'get_resized_image'. - In an embodiment, the container descriptor is created in a format that is supported by the type of container to be generated. For example, if such container type is Docker, the container descriptor format is a Dockerfile. A Dockerfile (container descriptor) is a script composed of various instructions and arguments that, when successively executed, perform actions on a container descriptor in order to generate a new container.
- At S360, a container for executing the received algorithm is generated using the container descriptor. As a non-limiting example, such container may be a Docker container. In an embodiment, the container is generated from a "base" container descriptor which defines mandatory resources for the proper execution of the algorithm. Referring back to the above example, the base container descriptor may define native Python and the new container would include the additional functions 'display' and 'get_resized_image'. In an embodiment, the user can edit or modify the container descriptor or any container generated thereof.
- The various embodiments have been discussed herein with a reference to a software container. The software container can be replaced with a virtual machine, a light virtual machine, or any applicable virtualization technologies without departing from the scope of the disclosure.
- The various embodiments disclosed herein can be implemented as hardware, firmware, software, or any combination thereof. Moreover, the software is preferably implemented as an application program tangibly embodied on a program storage unit or computer readable medium consisting of parts, or of certain devices and/or a combination of devices. The application program may be uploaded to, and executed by, a machine comprising any suitable architecture. Preferably, the machine is implemented on a computer platform having hardware such as one or more central processing units ("CPUs"), a memory, and input/output interfaces. The computer platform may also include an operating system and microinstruction code. The various processes and functions described herein may be either part of the microinstruction code or part of the application program, or any combination thereof, which may be executed by a CPU, whether or not such a computer or processor is explicitly shown. In addition, various other peripheral units may be connected to the computer platform such as an additional data storage unit and a printing unit. Furthermore, a non-transitory computer readable medium is any computer readable medium except for a transitory propagating signal.
- It should be understood that any reference to an element herein using a designation such as "first," "second," and so forth does not generally limit the quantity or order of those elements. Rather, these designations are generally used herein as a convenient method of distinguishing between two or more elements or instances of an element. Thus, a reference to first and second elements does not mean that only two elements may be employed there or that the first element must precede the second element in some manner. Also, unless stated otherwise a set of elements comprises one or more elements. In addition, terminology of the form "at least one of A, B, or C" or "one or more of A, B, or C" or "at least one of the group consisting of A, B, and C" or "at least one of A, B, and C" used in the description or the claims means "A or B or C or any combination of these elements." For example, this terminology may include A, or B, or C, or A and B, or A and C, or A and B and C, or 2A, or 2B, or 2C, and so on.
- All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the principles of the disclosed embodiments and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, aspects, and embodiments, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.
Claims (14)
- A computer-implemented method for automatically creating executable software code from a textual description of a computational algorithm, comprising:receiving (130, 132, 310) in electronic form, by a first electronic computing device, a textual description of a computational algorithm, the textual description being of insufficient quality to be successfully compiled as is to produce executable software code;generating (130, 132, 140) automatically, by a second electronic computing device, a software container based on the received textual description;instantiating (140) automatically, by a third electronic computing device, a computing environment, wherein the computing environment includes computing resources configured to support execution of the software container; andexecuting, by a fourth electronic computing device, the software container in the instantiated computing environment;wherein generating the software container further comprises:analyzing the received textual description to generate a dependency tree (S330);determining (S340), based at least on the generated dependency tree, a minimal set of functions and resources to be included in the software container, wherein the minimal set of functions are those needed for the proper execution of the software container's textual description in the computing environment;generating (S350) a container descriptor based on the determined minimal set of functions and resource whereby the container descriptor lists, at least, a minimal set of resources and their dependencies that should be included in the software container; andgenerating (S360) the software container using the container descriptor whereby the software container encapsulates all the software resources required for executing the algorithm.
- The method of claim 1, further comprising:
saving the software container in a data warehouse. - The method of claim 2, further comprising:generating metadata indicating at least the contents of the software container; andindexing the software container stored in the data warehouse using the generated metadata, thereby providing searchable software containers.
- The method of claim 1, wherein the textual description may be in a format of at least one of: pseudo code, a mathematical equation, a written description, and a code sample.
- The method of claim 1, wherein the textual description is extracted from at least one printed publication.
- The method of claim 1, wherein the textual description is uploaded by a user.
- The method of claim 1, further comprising:
benchmarking the computational algorithm based on the execution of the software container. - The method of claim 7, further comprising:providing a unified input for the execution of the software container, wherein the unified input is the same for all software containers being benchmarked; andmeasuring performance parameters during the execution of the software container, wherein each of the performance parameters is any of: agnostic to functions performed by the computational algorithm, and specific to at least one function performed by the computational algorithm.
- The method of claim 1, wherein at least two of the first, second, third, and fourth electronic computing devices are part of a single physical electronic computing device.
- The method of claim 1, further comprising:identifying a programming language for the computational algorithm; anddetermining the minimal set of functions and resources based on the identified programming language.
- The method of claim 1, wherein instantiating the computing environment further comprises:
instantiating the computing environment on-demand, wherein the instantiated computing environment includes resources selected based on the identified programming language. - The method of claim 1, further comprising:
caching resources produced during execution of the software container, wherein the resources include at least one of compilation files, installation files, and the results, wherein the cached resources are shared among a plurality of computing environments. - A non-transitory computer readable medium having stored thereon instructions for causing one or more processing units to execute a method for automatically creating executable software code from a textual description of a computational algorithm comprising the steps of:receiving (130, 132, 310) a textual description of a computational algorithm, the textual description being of insufficient quality to be successfully compiled as is to produce executable software code;generating (130, 132, 140) a software container based on the received textual description;instantiating (140) a computing environment on a computing device, wherein the computing environment includes computing resources configured to support execution of the software container; andexecuting the software container in the instantiated computing environment;wherein generating the software container further comprises:analyzing the received textual description to generate a dependency tree (S330);determining (S340), based at least on the generated dependency tree, a minimal set of functions and resources to be included in the software container, wherein the minimal set of functions are those needed for the proper execution of the software container's textual description in the computing environment;generating (S350) a container descriptor based on the determined minimal set of functions and resource whereby the container descriptor lists, at least, a minimal set of resources and their dependencies that should be included in the software container; andgenerating (S360) the software container using the container descriptor whereby the software container encapsulates all the software resources required for executing the algorithm.
- A system (100) for automatically creating executable software code from a textual description of a computational algorithm, comprising:a processing system (130, 132, 140); anda memory (130, 132, 140), the memory containing instructions that, when executed by the processing system, configure the system to:receive (130, 132, 310) a textual description of a computational algorithm, the textual description being of insufficient quality to be successfully compiled as is to produce executable software code;generate (130, 132, 140) a software container (133) based on the received textual description byanalyzing the received textual description to generate a dependency tree (S330);determining (S340), based at least on the generated dependency tree, a minimal set of functions and resources to be included in the software container, wherein the minimal set of functions are those needed for the proper execution of the software container's textual description in the computing environment;generating (S350) a container descriptor based on the determined minimal set of functions and resource whereby the container descriptor lists, at least, a minimal set of resources and their dependencies that should be included in the software container; andgenerating (S360) the software container using the container descriptor whereby the software container encapsulates all the software resources required for executing the algorithm;instantiate (140) a computing environment on a computing device, wherein the computing environment includes computing resources configured to support execution of the software container; andexecute the software container in the instantiated computing environment.
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201562181451P | 2015-06-18 | 2015-06-18 | |
US201562271800P | 2015-12-28 | 2015-12-28 | |
PCT/US2016/038050 WO2016205625A1 (en) | 2015-06-18 | 2016-06-17 | A computing platform and method thereof for searching, executing, and evaluating computational algorithms |
Publications (3)
Publication Number | Publication Date |
---|---|
EP3311265A1 EP3311265A1 (en) | 2018-04-25 |
EP3311265A4 EP3311265A4 (en) | 2019-02-20 |
EP3311265B1 true EP3311265B1 (en) | 2020-06-17 |
Family
ID=57546318
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP16812509.4A Active EP3311265B1 (en) | 2015-06-18 | 2016-06-17 | A computing platform and method thereof for searching, executing, and evaluating computational algorithms |
Country Status (3)
Country | Link |
---|---|
US (4) | US9965256B2 (en) |
EP (1) | EP3311265B1 (en) |
WO (2) | WO2016205625A1 (en) |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108628733B (en) * | 2017-03-16 | 2021-06-15 | 中国移动通信集团山西有限公司 | Method and device for testing batch service processing operation |
US10528366B2 (en) * | 2017-06-05 | 2020-01-07 | International Business Machines Corporation | Facilitating multi-inheritance within a single inheritance container-based environment |
US10310959B2 (en) * | 2017-11-07 | 2019-06-04 | Bank Of America Corporation | Pre-deployment validation system using intelligent databases |
US12118361B2 (en) * | 2018-11-21 | 2024-10-15 | Sri International | Unicontainers |
US11023356B2 (en) | 2019-01-03 | 2021-06-01 | International Business Machines Corporation | Utilization of publicly available source code |
US20220404804A1 (en) * | 2021-06-16 | 2022-12-22 | Fisher-Rosemount Systems, Inc. | Security Services in a Software Defined Control System |
Family Cites Families (28)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6314457B1 (en) | 1999-04-21 | 2001-11-06 | Airclic, Inc. | Method for managing printed medium activated revenue sharing domain name system schemas |
US6775824B1 (en) * | 2000-01-12 | 2004-08-10 | Empirix Inc. | Method and system for software object testing |
WO2001077882A1 (en) | 2000-04-07 | 2001-10-18 | Healthmedia, Inc. | System and method for generating computer source code |
US20030005407A1 (en) * | 2000-06-23 | 2003-01-02 | Hines Kenneth J. | System and method for coordination-centric design of software systems |
FR2811093A1 (en) | 2000-06-30 | 2002-01-04 | St Microelectronics Sa | DEVICE AND METHOD FOR EVALUATING ALGORITHMS |
US20020069240A1 (en) | 2000-12-06 | 2002-06-06 | Berk Donald J. | Method and apparatus for electronically updating printed publications |
US7159224B2 (en) * | 2002-04-09 | 2007-01-02 | Sun Microsystems, Inc. | Method, system, and articles of manufacture for providing a servlet container based web service endpoint |
CA2418670A1 (en) | 2003-02-11 | 2004-08-11 | Ibm Canada Limited - Ibm Canada Limitee | Method and system for generating executable code for formatiing and printing complex data structures |
US8527943B1 (en) * | 2004-02-12 | 2013-09-03 | Raju V. Chiluvuri | System and method of application development |
WO2005114479A2 (en) | 2004-05-21 | 2005-12-01 | Computer Associates Think, Inc. | Automated creation of web page to xml translation servers |
US7797687B2 (en) * | 2005-08-04 | 2010-09-14 | Microsoft Corporation | Parameterized unit tests with behavioral purity axioms |
US20070150821A1 (en) | 2005-12-22 | 2007-06-28 | Thunemann Paul Z | GUI-maker (data-centric automated GUI-generation) |
US20070272733A1 (en) | 2006-05-25 | 2007-11-29 | Donald Spector | Printed publication with a readable code for connection to a computing device |
US7681045B2 (en) * | 2006-10-12 | 2010-03-16 | Black Duck Software, Inc. | Software algorithm identification |
US8510712B1 (en) * | 2007-03-19 | 2013-08-13 | Interactive TKO, Inc. | Testing in-container software objects |
US20090108057A1 (en) * | 2007-10-24 | 2009-04-30 | Hong Mu | Using Quick Response Codes to Provide Interactive Services |
US8120811B2 (en) | 2007-11-21 | 2012-02-21 | Quad/Graphics, Inc. | System and method for adding data to a printed publication |
US8954869B2 (en) | 2007-12-17 | 2015-02-10 | International Business Machines Corporation | Generating a front end graphical user interface for a plurality of text based commands |
US8373724B2 (en) | 2009-01-28 | 2013-02-12 | Google Inc. | Selective display of OCR'ed text and corresponding images from publications on a client device |
CA2684348A1 (en) | 2009-11-04 | 2011-05-04 | Ibm Canada Limited - Ibm Canada Limitee | Algorithm complexity identification |
US8756571B2 (en) | 2010-05-07 | 2014-06-17 | Hewlett-Packard Development Company, L.P. | Natural language text instructions |
FR2964224A1 (en) * | 2010-08-24 | 2012-03-02 | Thales Sa | METHOD AND DEVICE FOR DEPLOYING AND ASSISTING THE DEPLOYMENT OF COMPONENTS FORMING A REAL TIME SYSTEM ONBOARD |
EP2630580A4 (en) | 2010-10-19 | 2014-04-02 | Hewlett Packard Development Co | Methods and systems for generation of authorized virtual appliances |
US8826068B2 (en) * | 2011-11-23 | 2014-09-02 | Microsoft Corporation | Automated testing of applications in cloud computer systems |
GB2501757A (en) * | 2012-05-04 | 2013-11-06 | Ibm | Instrumentation of software applications for configuration thereof |
US9208307B2 (en) | 2013-08-08 | 2015-12-08 | Raytheon Company | Automatic algorithm discovery using reverse dataflow analysis |
US9170808B2 (en) * | 2013-11-07 | 2015-10-27 | Sap Se | Dynamic containerization |
US20160026442A1 (en) * | 2014-07-25 | 2016-01-28 | Mahesh Kumar Chhaparia | Method and system for hosting and running user configured dynamic applications through a software container on a computing device |
-
2016
- 2016-06-17 WO PCT/US2016/038050 patent/WO2016205625A1/en active Application Filing
- 2016-06-17 WO PCT/US2016/038053 patent/WO2016205628A1/en active Application Filing
- 2016-06-17 EP EP16812509.4A patent/EP3311265B1/en active Active
- 2016-06-17 US US15/185,538 patent/US9965256B2/en active Active
- 2016-06-17 US US15/185,513 patent/US9971573B2/en active Active
-
2018
- 2018-05-07 US US15/972,978 patent/US10489124B2/en active Active
- 2018-05-09 US US15/975,368 patent/US10496380B2/en active Active
Non-Patent Citations (1)
Title |
---|
None * |
Also Published As
Publication number | Publication date |
---|---|
US10496380B2 (en) | 2019-12-03 |
WO2016205625A1 (en) | 2016-12-22 |
US20180253286A1 (en) | 2018-09-06 |
US20180260194A1 (en) | 2018-09-13 |
WO2016205628A1 (en) | 2016-12-22 |
US10489124B2 (en) | 2019-11-26 |
US9965256B2 (en) | 2018-05-08 |
US20160371176A1 (en) | 2016-12-22 |
US9971573B2 (en) | 2018-05-15 |
EP3311265A4 (en) | 2019-02-20 |
EP3311265A1 (en) | 2018-04-25 |
US20160371175A1 (en) | 2016-12-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3311265B1 (en) | A computing platform and method thereof for searching, executing, and evaluating computational algorithms | |
US10152309B2 (en) | Cross-library framework architecture feature sets | |
US9286042B2 (en) | Control flow graph application configuration | |
Garbervetsky et al. | Toward full elasticity in distributed static analysis: The case of callgraph analysis | |
US10698666B2 (en) | Automatically building software projects | |
US12045589B2 (en) | Software development improvement stage optimization | |
US10635483B2 (en) | Automatic synopsis generation for command-line interfaces | |
Fernández-Prades et al. | Continuous reproducibility in GNSS signal processing | |
Pearce et al. | Towards collaborative continuous benchmarking for HPC | |
US11315436B2 (en) | Method and system for automatically providing graphical user interfaces for computational algorithms described in printed publications | |
KR20130020135A (en) | System and method of providing the developer list of developing code simultaneously in an integrated development environment | |
Aumüller et al. | Running experiments with confidence and sanity | |
Abate et al. | Adoption of academic tools in open source communities: the Debian case study | |
Salohonka | Automated testing of React Native applications | |
WO2018022430A1 (en) | Method and system for automatically providing graphical user interfaces for computational algorithms described in printed public | |
US20240272890A1 (en) | Automated deployment of continuous integration | |
US20240319992A1 (en) | Utilizing multiple analyses to migrate an application to a cloud computing environment | |
Asegehegn | Evaluation of Rust and WebAssembly when building a Progressive Web Application: An analysis of performance and memory usage | |
US20240329983A1 (en) | Development environment integrated with infrastructure cost estimation system | |
Khan | C# 7 and. NET Core 2.0 High Performance: Build highly performant, multi-threaded, and concurrent applications using C# 7 and. NET Core 2.0 | |
Barba et al. | Comparative Analysis of OpenACC Compilers | |
Heirendt et al. | ARTENOLIS: Automated Reproducibility and Testing Environment for Licensed Software | |
Aley et al. | Development Tools | |
Morais | Automatic Refactoring for Energy Efficiency in Continuous Integration Pipelines | |
Riehl et al. | KBase: A Platform for Reproducible Bioinformatics Research |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20171221 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
A4 | Supplementary search report drawn up and despatched |
Effective date: 20190123 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G06F 9/44 20180101AFI20190117BHEP Ipc: G06F 11/34 20060101ALI20190117BHEP Ipc: G06F 8/36 20180101ALI20190117BHEP Ipc: G06F 9/455 20180101ALI20190117BHEP |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20190905 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20200131 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602016038394 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: NV Representative=s name: COSMOVICI INTELLECTUAL PROPERTY SARL, CH Ref country code: AT Ref legal event code: REF Ref document number: 1282173 Country of ref document: AT Kind code of ref document: T Effective date: 20200715 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200917 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200918 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20200617 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200917 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1282173 Country of ref document: AT Kind code of ref document: T Effective date: 20200617 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201019 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201017 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602016038394 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200617 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20200630 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200617 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 |
|
26N | No opposition filed |
Effective date: 20210318 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200630 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200617 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: CH Payment date: 20230702 Year of fee payment: 8 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20240627 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20240627 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20240625 Year of fee payment: 9 |