WO2019222941A1 - 用于评估应用部署的方法、设备、计算机程序产品和可读介质 - Google Patents

用于评估应用部署的方法、设备、计算机程序产品和可读介质 Download PDF

Info

Publication number
WO2019222941A1
WO2019222941A1 PCT/CN2018/088029 CN2018088029W WO2019222941A1 WO 2019222941 A1 WO2019222941 A1 WO 2019222941A1 CN 2018088029 W CN2018088029 W CN 2018088029W WO 2019222941 A1 WO2019222941 A1 WO 2019222941A1
Authority
WO
WIPO (PCT)
Prior art keywords
application
physical device
test case
physical
resources
Prior art date
Application number
PCT/CN2018/088029
Other languages
English (en)
French (fr)
Inventor
马万里
袁勇
Original Assignee
西门子股份公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 西门子股份公司 filed Critical 西门子股份公司
Priority to PCT/CN2018/088029 priority Critical patent/WO2019222941A1/zh
Priority to CN201880089377.5A priority patent/CN111712795A/zh
Priority to EP18919504.3A priority patent/EP3754489A4/en
Priority to US16/975,792 priority patent/US20210248056A1/en
Publication of WO2019222941A1 publication Critical patent/WO2019222941A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/3604—Software analysis for verifying properties of programs
    • G06F11/3608—Software analysis for verifying properties of programs using formal methods, e.g. model checking, abstract interpretation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/60—Software deployment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/3668—Software testing
    • G06F11/3672—Test management
    • G06F11/3684—Test management for test design, e.g. generating new test cases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/3668—Software testing
    • G06F11/3672—Test management
    • G06F11/3692—Test management for test results analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/60—Software deployment
    • G06F8/61—Installation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network

Definitions

  • Embodiments of the present disclosure relate to the field of computers, and more particularly, to methods, devices, computer program products, and readable media for evaluating application deployment.
  • a physical device such as a computer
  • a physical device is typically deployed with one or more applications to perform corresponding functions.
  • the user can request an application provider such as an application server to download and install more applications on the physical device, so that the device can use the application to perform more desired functions.
  • an application server directly deploys an application to a corresponding device in response to a user request.
  • newly deployed applications may affect the normal operation of deployed applications on the device. This is undesirable in many scenarios, especially if the affected application is responsible for the core functions of the device.
  • application designers are generally required to predict the operating environment of the application as much as possible during development to avoid conflicts with other applications in different environments. Such laboratory predictions are often limited and cannot test all real-world operating environments. Therefore, in actual deployment, adverse effects of one application on another application still occur from time to time.
  • a possible solution to avoid such adverse effects is that the user actively removes new applications from the device or prohibits the operation of new applications after discovering that existing applications are affected, thereby ensuring that applications responsible for important functions can run normally.
  • Embodiments of the present disclosure provide such a scheme for pre-evaluating application deployment.
  • a method for evaluating application deployment includes, in response to a request to deploy a first application to a physical device, obtaining application information related to the first application and device information related to the physical device, the device information indicating at least a second application that has been deployed on the physical device; based on Application information and device information to create a simulation environment, the simulation environment simulates the operating environment of the physical device after the first application and the second application are deployed; and the first application and the second application are obtained by running the first application and the second application in the simulation environment Associated evaluation results that indicate the impact of deploying the first application into a physical device on the second application.
  • obtaining an evaluation result associated with the first application includes determining a first test case and a second test case corresponding to the first application and a second application, respectively, and the first test case and the second test case are respectively Specify test inputs, operating conditions, and expected outputs for the first and second applications.
  • a test plan is generated based on the first test case and the second test case, and the test plan specifies a running process of the first test case and the second test case in a simulation environment. Run the test plan in a simulated environment to obtain the evaluation results.
  • At least one of the first test case and the second test case is pre-created and stored. This can avoid the time-consuming design of test cases in real time, and the test cases corresponding to the same application can be reused.
  • the physical device is a first physical device
  • creating the simulated environment includes configuring a second physical device as the simulated environment, the second physical device being different from the first physical device.
  • Using a separate physical device for the purpose of application evaluation can avoid the impact of application evaluation on the physical device of interest.
  • separate physical devices can be reused for deployment evaluations for different physical devices or different applications.
  • configuring the second physical device includes deploying the first application and the second application to the second physical device, and configuring the second physical device based on the application information and the device information.
  • the second physical device can be configured in software and hardware performance similar to that of the first physical device after the first application and the second application are deployed, so that it can be more accurately and reliably used for evaluation Impact of the first application on the second application after it is deployed.
  • the application information indicates the identity of the first application and the resource requirements of the first application.
  • the device information indicates available resources and unoccupied resources of the physical device, and the available resources include at least one of computing resources, storage resources, and communication resources. Both such application information and / or device information may help to create a simulated environment similar to a real environment possible on a physical device.
  • obtaining the evaluation results associated with the first application includes determining whether the second application will experience a resource shortage or an operation failure by running the first application and the second application in a simulated environment, and in response to determining the first application A shortage of resources or an operation failure may occur in the second application, and an evaluation result is generated to indicate that deploying the first application on a physical device will affect the second application.
  • the impact of the first application on the second application may be reflected in the occupation of resources on the physical device, which causes the second application to have insufficient running resources or affects the normal operation of the second application.
  • the method further includes, in response to the evaluation result indicating that the deployment of the first application into the physical device has no impact on the second application, so that the first application is deployed to the physical device.
  • causing the first application to be deployed to the physical device includes triggering the first application to be downloaded from the application server to the physical device. Therefore, in a case where it is confirmed that the deployment of the first application is secure, the first application can be actually deployed to a physical device (for example, downloaded from an application server to the physical device).
  • the physical device is included in an Internet of Things (IoT) system. Therefore, it is possible to provide favorable evaluation of risky application deployment for the devices in the IoT system, and eliminate the requirement for manual monitoring.
  • IoT Internet of Things
  • an electronic device in a second aspect of the present disclosure, includes a processor and a memory coupled to the processor.
  • the memory has instructions stored therein, and the instructions, when executed by the processor, cause the electronic device to perform actions.
  • the actions include, in response to a request to deploy the first application to a physical device, obtaining application information related to the first application and device information related to the physical device, the device information indicates at least a second application that has been deployed on the physical device; based on the application Information and device information to create a simulation environment, which simulates the operating environment of the physical device after the first and second applications are deployed; and obtains the first application by running the first and second applications in the simulation environment
  • the combined evaluation results indicate the impact of deploying the first application to the physical device on the second application.
  • a device for evaluating application deployment including a module for performing steps in various embodiments of the method according to the first aspect.
  • a computer program product is provided that is tangibly stored on a computer-readable medium and includes computer-executable instructions that, when executed, cause at least one processor to execute a program based on Various embodiments of the method of the first aspect.
  • a computer-readable medium having computer-executable instructions stored thereon that, when executed, cause at least one processor to perform various embodiments of the method according to the first aspect.
  • Figure 1 shows a schematic diagram of an environment for conventional application deployment
  • FIG. 2 illustrates a schematic diagram of an application deployment environment according to an embodiment of the present disclosure
  • FIG. 3 shows a flowchart of a process of evaluating application deployment according to one embodiment of the present disclosure
  • FIG. 4 illustrates a schematic diagram of an application deployment environment according to another embodiment of the present disclosure
  • 5A illustrates a flowchart of a process for simulating application execution according to an embodiment of the present disclosure
  • 5B shows a flowchart of a process for generating and utilizing evaluation results according to an embodiment of the present disclosure
  • FIG. 6 illustrates a schematic diagram of an application deployment environment according to yet another embodiment of the present disclosure.
  • FIG. 7 illustrates a block diagram of an example device that can be used to implement embodiments of the present disclosure.
  • the term “including” and variations thereof mean open-ended inclusion, that is, “including but not limited to.” Unless specifically stated otherwise, the term “or” means “and / or”. The term “based on” means “based at least in part on.” The terms “one example embodiment” and “one embodiment” mean “at least one example embodiment.” The term “another embodiment” means “at least one further embodiment”. The terms “first”, “second”, etc. may refer to different or the same objects. Other explicit and implicit definitions may be included below.
  • FIG. 1 illustrates an environment 100 for such conventional application deployment.
  • the environment 100 includes an application server 110 and a physical device 120.
  • the application server 110 is deployed in the cloud 130.
  • the cloud 130 may be a public cloud or a private cloud, which is configured to centrally manage and utilize software and hardware resources deployed therein to provide organizations and / or individuals with a variety of functions such as data storage, sharing, computing, and analysis And / or services.
  • the cloud 130 may also be referred to as a cloud-based network, in which software and / or hardware resources may be interconnected and may also be accessed by external devices and users.
  • the environment 130 based on the cloud 130 can realize real-time communication of users, devices, and products, and build a highly flexible personalized, digital intelligent ecosystem.
  • the application server 110 is configured to manage various applications.
  • an application refers to software that is written to implement a specific function. When installed on a specific device, the application can be run to achieve corresponding functions, such as data acquisition control, data processing, mathematical operations, process management, social functions, and so on. It should be understood that the examples given here are only some non-limiting examples of the functions provided by the application, and the application can be designed to implement a variety of desired functions as needed.
  • the applications managed by the application server 110 may be provided by multiple sources (eg, different developers, application providers, etc.).
  • the application server 110 may maintain installation files and application profiles of these applications, and control uploading, compliance, downloading, updating, etc. of these applications. Therefore, the application server 110 may also be referred to as an application store.
  • the user 102 may request from the application server 110 to request the deployment of a specific application to the physical device 120.
  • the user 102 may send a request to the application server 110 via, for example, an input device (eg, a user interface) of the application server 110 or via a user terminal device.
  • the user 102 may also be a user of the physical device 120.
  • the application server 110 downloads a corresponding application to the physical device 120.
  • the physical device 120 can run the application, thereby utilizing the application to implement a corresponding function.
  • the physical device 120 can access the cloud 130, particularly the application server 110 therein.
  • the physical device 120 may be any device having computing and communication capabilities.
  • the physical device 120 may be any type of fixed device, mobile device, or portable device, including a desktop computer, laptop computer, notebook computer, netbook computer, tablet computer, multimedia computer, multimedia tablet, Internet node, Mobile phones, sites, IoT devices, and more.
  • the physical device 120 may be a user terminal, or may be a device arranged on an industrial site, such as a control device for a motor, a sensor, and other devices, a device for data analysis and calculation, a device for controlling and transmitting job instructions, and the like Wait.
  • the cloud 130 including the application server 110 and the devices connected to it can be referred to as an Internet of Things (IoT) system.
  • IoT Internet of Things
  • the application server 110 may also respond to requests from multiple users for deploying the same application or different applications to different physical devices.
  • the application server 110 is shown in the cloud 130, other software and / or hardware resources may also be deployed in the cloud 130.
  • the scope of the present disclosure is not limited in this regard.
  • the implementation of the present disclosure is not actually limited to this, but can be implemented in other different types of environments (e.g., environments not related to cloud-based networks) in).
  • an application requested by a user is directly deployed to a physical device, for example, installed and run on the physical device.
  • newly deployed applications may affect the normal operation of deployed applications on the device. This would be undesirable if the affected applications were responsible for the core functions of the device.
  • users can also remove (for example, uninstall) a newly deployed application when it is found that an important application is affected, or prohibit the operation of the application, as mentioned above, this not only requires users to pay attention to the running status of the application in real time after application deployment And it is likely to cause unnecessary losses due to the failure to detect the impact of the application in time.
  • real-time attention to the operating status may be difficult to achieve, or the discovery of adverse effects on the application may be delayed. For example, in industrial sites, the number of devices to deploy applications is large, and real-time monitoring is not supported by the device installation location. These reasons may cause users to remove or disable undesired applications in a timely manner.
  • a scheme for evaluating application scheduling is proposed. This approach requires that the impact of the application on the deployed application on the physical device be evaluated before the requested application is deployed to the physical device. The results of such an assessment can be used to assist in determining whether the requested application should actually be deployed to a physical device, thereby avoiding risky application deployment.
  • FIG. 2 illustrates a schematic diagram of an application deployment environment 100 according to one embodiment of the present disclosure.
  • an evaluation device 202 is provided, and is used when the application server 110 receives a request from a user 102 to deploy an application to the physical device 120, Assess the impact of the application on deployed applications on the physical device 120.
  • the evaluation device 202 is configured to be communicatively connected with the application server 110 and the physical device 120 for receiving information from the application server 110 and the physical device 120 and, in some cases, sending information to the application server 110.
  • Such a communication connection may be a wired or wireless connection.
  • the evaluation device 202 may include a device or server with communication and computing capabilities. It should be understood that although shown as a separate device, the evaluation device 202 may be deployed in the cloud 130 or integrated into the application server 110.
  • FIG. 3 illustrates a flowchart of a process 300 for evaluating application deployment according to one embodiment of the present disclosure.
  • the process 300 may be implemented by the evaluation device 202.
  • the evaluation device 202 obtains application information related to one application (referred to as a first application for convenience of description) and device information related to the physical device 120.
  • the acquisition of the application information and the device information may be in response to a request to deploy the first application to the physical device 120.
  • the application server 110 may actively send application information of the first application to the evaluation device 202, and indicate to the evaluation device 202 that it is expected to evaluate the first application on the physical device. Impact of deployed applications on 120.
  • the evaluation device 202 receives application information from the application server 110 and requests and receives device information from the physical device 120.
  • the application information related to the first application may also be referred to as an application profile of the first application, and is used to describe one or more aspects of the first application.
  • the first application-related application information indicates an identification of the first application.
  • the identifier may be used in the application server 110 and the physical device 120 to distinguish the first application.
  • the application information may also indicate operating requirements, such as resource requirements, of the first application. When running on a device, an application will typically require a certain amount of resources on the device. In some cases, a certain amount of resources and the right type of resources can support the normal operation of the application. Therefore, the application information may indicate the requirements of the first application on the types and / or resource amounts of various resources.
  • the resources involved in running the first application may include one or more of computing resources, storage resources, and communication resources.
  • Computing resources such as the processor of the device include a central processing unit (CPU), a dedicated processing unit such as a graphics processing unit (GPU), a field programmable gate array (FPGA), and the like.
  • Storage resources include the device's memory, flash memory, and persistent storage space.
  • the communication resources may include, for example, a general-purpose input / output (I / O) interface, a device-internal communication resource such as a bus, and / or a device-external communication resource such as an Ethernet port.
  • the device information related to the physical device 120 describes one or more aspects of the physical device 120.
  • the device information indicates at least an identification of an application (called a second application for convenience of description) that has been deployed on the physical device 120.
  • the identification may be used in the application server 110 and the physical device 120 to distinguish the second application.
  • the physical device 120 may be deployed with multiple applications.
  • the device information may include identifications of a plurality of applications.
  • the device information may include an identification of one or more applications responsible for critical tasks on the physical device 120. For example, if the physical device 120 is configured to perform machine control. To achieve this task, an application for sensor control and an application for sensor data analysis are deployed on the physical device 120.
  • the device information may include the identities of the two applications.
  • the device information may also indicate the resource status of the physical device 120, such as the available resources and unoccupied resources of the physical device 120.
  • the available resources refer to the resources that the physical device 120 can actually dispose of, including one or more of computing resources, storage resources, and communication resources.
  • the unoccupied resources refer to the unoccupied resource portion of the actual disposable resources, because a portion of the resources of the physical device 120 may be occupied by deployed applications and / or operating systems.
  • the application information and device information may include other related information of the first application and the physical device.
  • the application information may further include requirements of the first application on an operating system type, a network type, and the like.
  • the device information may also include information about the operating system type, device type, and device model of the physical device.
  • the evaluation device 202 creates a simulation environment based on the obtained application information and device information.
  • the simulation environment simulates an operating environment of the physical device 120 after the first application and the second application are deployed.
  • a simulation environment independent of the actual operating environment of the physical device 120 may be created for evaluation of the first application.
  • the simulation environment may be created as similar as possible to the operating environment of the physical device 120 after the first application and the second application are deployed.
  • the simulation environment is similar to the operating environment of the physical device 120 in terms of application deployment, hardware conditions, and network conditions.
  • a simulation environment may be created on the evaluation device 202 by utilizing resources (including computing, storage, and communication resources, etc.) that the evaluation device 202 can provide.
  • the evaluation device 202 may utilize another physical device to create a simulated environment.
  • the physical device 120 is sometimes referred to as a first physical device, and the physical device used to create the simulated environment may sometimes be referred to as a second physical device.
  • the second physical device is different from the first physical device to avoid the impact of the application evaluation on the first physical device.
  • a separate second physical device can be reused for deployment evaluations for different physical devices or different applications.
  • FIG. 4 illustrates one example of an environment 100 according to such an embodiment.
  • a physical device 410 is provided as a second physical device.
  • the physical device 410 is configured to be communicatively connected with the evaluation device 202 and the application server 110.
  • the evaluation device 202 configures the physical device 410 as a simulation environment based on the application information and the device information.
  • the physical device 410 will be configured to be the same or similar to the physical device 120 in a number of ways, such as the same or similar in terms of operating system, storage, and / or computing power.
  • the physical device 410 may even be selected as a device of the same model as the physical device 120.
  • the evaluation device 202 may determine the simulation requirements of the physical device 410 in various aspects such as application deployment, resource type, available resource amount, and operating system type based on the application information and the device information.
  • the evaluation device 202 may deploy the first application and the second application to the physical device 410.
  • the evaluation device 202 may provide the identities of the first application and the second application to the physical device 410, and trigger the physical device 410 to download the first application and the second application from the application server 110.
  • the evaluation device 202 may also request the application server 110 to directly deploy the first application and the second application to the physical device 410.
  • the evaluation device 202 also configures the second physical device based on the application information and device information, including configuring the amount and type of resources available to the physical device 410 in the subsequent application evaluation process, and the operating system that the second physical device presents to the deployed application Type, device type, and so on.
  • the evaluation device 202 can convert the application information and device information into corresponding configuration parameters and provide these parameters to the physical device 410.
  • the physical device 410 may adjust the configuration of the device itself based on the received parameters, thereby simulating the operating environment of the physical device 120.
  • the evaluation device 202 obtains an evaluation result associated with the first application by running the first application and the second application in a simulated environment.
  • the evaluation result indicates the impact of deploying the first application into the physical device 120 on the second application.
  • FIG. 5A illustrates a flowchart of a process 502 for application deployment evaluation according to another embodiment of the present disclosure.
  • Process 502 may be considered as an example of obtaining evaluation results in process 300.
  • the process 502 may also be implemented by the evaluation device 202.
  • the evaluation device 202 determines a first test case and a second test case corresponding to the first application and the second application, respectively.
  • the evaluation device 202 may provide the determined test case to the physical device 410.
  • the evaluation device 202 may also serve as a control device, triggering the physical device 410 to determine a test case of the first application and the second application.
  • the first test case and the second test case specify test inputs, operating conditions, and expected outputs of the first and second applications, respectively.
  • the first application and the second application may not be able to obtain real application inputs, so test inputs and corresponding operating conditions can be given by designing test cases.
  • test cases under specific operating conditions, test inputs are provided to the corresponding applications, and the output after the application is recorded is recorded. If the output is the same as expected, the application is considered to be running normally; otherwise, it can be determined that the application is in error.
  • the design of test cases can be implemented using any currently known or future technology to be developed in the field of software testing, and the scope of this disclosure is not limited in this respect.
  • Test cases with the first application and / or the second application may be pre-created and stored, such as in a repository. Because the applications provided by the application server 110 can be deployed to multiple different physical devices, by creating and storing test cases for these applications in advance, time-consuming design of test cases can be avoided, and pre-stored test cases can be reused for Different application deployment evaluation processes.
  • FIG. 6 illustrates one example of an environment 100 in which pre-created test cases can be provided.
  • a storage repository 610 is provided for storing test cases created in advance and corresponding to respective applications.
  • the repository 610 is shown as being communicatively connected with the physical device 410.
  • the repository 610 may be deployed in the cloud 130 as part of the cloud 130.
  • the physical device 410 may obtain test cases of the first application and / or the second application from the repository 610 in response to a trigger of the evaluation device 202.
  • the repository 610 may alternatively or additionally be configured to be communicatively connected with the evaluation device 202. In such an embodiment, test cases may be obtained by the evaluation device 202 from the repository 610 and provided to the device 410.
  • the repository 610 stores the test case with the identity of the corresponding application.
  • the physical device 410 or the evaluation device 202 may obtain the first test case and / or the second test case from the repository 610 based on the identification of the first application and / or the second application.
  • the repository 610 may further include more test cases corresponding to other applications.
  • the evaluation device 202 generates a test plan based on the first test case and the second test case.
  • the test plan specifies the first test case and the second test case to run in a simulated environment.
  • the test plan may specify the running time of the first test case and the second test case during the entire running process, the operating range, the processing in the case of an output error, and so on.
  • the generation of the test plan may also be implemented using any currently known or future technology to be developed in the field of software testing, and the scope of the present disclosure is not limited in this respect.
  • test cases in addition to providing test cases that can correspond to applications, additional test cases can be provided for monitoring the operating system, availability of various resources simulated by the simulated environment (such as physical device 410), and so on. . These test cases can also be used to generate test plans. Thus, a comprehensive test of the physical device 120 can be achieved.
  • the evaluation device 202 runs a test plan in a simulated environment to obtain an evaluation result.
  • the evaluation device 202 may provide the generated test plan to the physical device 410 to run the test plan by the physical device 410 having the first and second applications installed, The operation result of the test plan is obtained from the physical device 410.
  • the evaluation device 202 may also act as a control device, triggering the physical device 410 to generate and run a test plan. During the execution of the test plan, the first application and the second application run according to their respective test cases.
  • the first application or the second application is input to the test input in the first test case or the second test case under specific operating conditions, and the corresponding output will be recorded for use with the expected output specified in the test case Compare.
  • the output during the test did not match the expected output, which means that the results of the application did not meet expectations.
  • the evaluation device 202 may determine an evaluation result associated with the first application based on a running result of the test plan. This can be judged from one or more aspects of the results of the operation. Specifically, based on the running result, it is determined whether the second application is running normally, thereby determining whether deploying the first application on the physical device by the first application will affect the second application. In some embodiments, if the evaluation device 202 needs to evaluate two or more applications on the physical device 120 and the operation result indicates that one or more of the applications are affected, the evaluation device 202 generates a corresponding evaluation result to indicate The impact of the deployment of an application on these specific applications.
  • FIG. 5B illustrates a process 504 of generating and utilizing evaluation results based on a test plan after the process 502 of FIG. 5A.
  • the process 504 may also be implemented by the evaluation device 202.
  • the evaluation device 202 determines an evaluation result by determining whether a resource shortage or an operation failure may occur in the second application.
  • the impact of the first application on the second application may be reflected in the occupation of resources on the physical device, which causes the second application to have insufficient running resources or affects the normal operation of the second application.
  • the first application may require more computing resources or occupy more network connection ports, so that the second application cannot occupy more resources.
  • the shortage of resources may affect the efficiency of the second application operation, which is not desirable.
  • the running process of the first application may conflict with the running process of the second application, thereby causing the running of the second application to fail. These conditions can be determined from the results of running the test plan.
  • the evaluation device 202 If it is determined in block 540 that the resource requirements of the second application can be met and the running results are correct (for example, the same or similar to the expected output of the test case), in block 550, the evaluation device 202 generates an evaluation result to instruct the first application to be deployed Going into the physical device 120 has no effect on the second application. Then, at block 560, the evaluation device 202 causes the first application to be deployed to the physical device 120. In some embodiments, the evaluation device 202 may trigger the first application to be downloaded from the application server to the physical device. The evaluation device 202 may send the application server 110 to the application server 110 an evaluation result indicating no impact or a confirmation response regarding application deployment.
  • the application server 110 downloads the first application to the physical device 120. Therefore, in a case where it is confirmed that the deployment of the first application is secure, the first application can be actually deployed to a physical device (for example, downloaded from an application server to the physical device).
  • the evaluation device 202 If it is determined in block 540 that a resource shortage or a running failure will occur in the second application, in block 570, the evaluation device 202 generates an evaluation result to indicate that the deployment of the first application on the physical device 120 will affect the second application. In this case, the evaluation device 202 prevents the first application from being deployed to the physical device 120 at 580. In some embodiments, the evaluation device 202 may send an unaffected evaluation result or a rejection response to the application deployment to the application server 110. In response to receiving such an evaluation result or response, the application server 110 may not deploy the first application to the physical device 120.
  • the application server 110 may also provide the user 102 with an indication that the first application cannot be deployed, and may also provide the user 102 with the reason that the second application in the physical device 120 will be affected. Identification about the affected second application may be provided by the evaluation device 202.
  • FIG. 7 illustrates a schematic block diagram of an example device 700 that can be used to implement embodiments of the present disclosure.
  • the device 700 may be used to implement the process 300 of FIG. 3, the process 502 of FIG. 5A, and / or the process 502 of FIG. 5B.
  • the device 700 may be implemented as the evaluation device 202, the physical device 120, or the physical device 410 described above.
  • the device 700 includes a central processing unit (CPU) 701, which can be loaded into a computer in a random access memory (RAM) 703 according to computer program instructions stored in a read only memory (ROM) 702 or from a storage unit 708 Program instructions to perform various appropriate actions and processes.
  • RAM random access memory
  • ROM read only memory
  • RAM 703 various programs and data required for the operation of the device 700 can also be stored.
  • the CPU 701, ROM 702, and RAM 703 are connected to each other through a bus 704.
  • An input / output (I / O) interface 705 is also connected to the bus 704.
  • the I / O interface 705 including: an input unit 706, such as a keyboard, a mouse, etc .; an output unit 707, such as various types of displays, speakers, etc .; a storage unit 708, such as a magnetic disk, an optical disk, etc. And a communication unit 709, such as a network card, a modem, a wireless communication transceiver, and the like.
  • the communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
  • the processing unit 701 performs various methods and processes described above, such as the process 300, the process 502, and / or the process 504.
  • process 300, process 502, and / or process 504 may be implemented as a computer software program or computer program product that is tangibly embodied on a computer-readable medium, such as a non-transitory computer-readable medium (Eg, storage unit 708).
  • a computer-readable medium such as a non-transitory computer-readable medium (Eg, storage unit 708).
  • part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709.
  • the CPU 701 may be configured to perform the process 300, the process 502, and / or the process 504 in any other suitable manner (eg, by means of firmware).
  • the above steps of the method of the present disclosure can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Ground, they can be implemented with program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or The steps are made into a single integrated circuit module for implementation. As such, the present disclosure is not limited to any particular combination of hardware and software.
  • certain embodiments of the present disclosure also include various program modules and / or integrated circuit modules for performing one or more steps of process 300, process 502, and / or process 504 and / or in other embodiments of the present disclosure Describe one or more other steps.
  • These program modules may be included or embodied in a device, such as the device 700 of FIG. 7.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Stored Programmes (AREA)

Abstract

用于评估应用部署的方法、设备、计算机程序产品和可读介质。在评估应用部署的方法中，响应于将第一应用部署到物理设备的请求，获得与第一应用相关的应用信息和与物理设备相关的设备信息（310），设备信息至少指示已部署在物理设备上的第二应用；基于应用信息和设备信息来创建模拟环境（320），模拟环境模拟物理设备在第一应用和第二应用被部署后的运行环境；通过在模拟环境中运行第一应用和第二应用来获取与第一应用相关联的评估结果（330），评估结果指示将第一应用部署到物理设备中对第二应用的影响。所述方法能够主动且有效地避免有风险的应用部署，并且免除了对人工监测的要求。

Description

用于评估应用部署的方法、设备、计算机程序产品和可读介质 技术领域
本公开的实施例涉及计算机领域，并且更具体地，涉及用于评估应用部署的方法、设备、计算机程序产品和可读介质。
背景技术
物理设备(例如计算机)通常被部署一个或多个应用，以执行相应的功能。根据需要，用户可以向例如应用服务器等应用提供商请求将更多应用下载和安装到物理设备上，从而使设备能够利用应用来执行更多期望的功能。
通常，应用服务器响应于用户请求而直接将应用部署到相应设备。然而，新部署的应用可能会影响设备上已部署的应用的正常运行。这在很多场景下是不期望的，特别是如果受影响的应用需负责该设备的核心功能。针对这个问题，一般要求应用设计者在开发期间尽可能预测应用的运行环境，以避免在不同环境中与其他应用的冲突。这样的实验室预测往往是有限的，无法测试全部的真实运行环境。因而，在实际部署中，一个应用对另一应用的不利影响仍时有发生。避免这种不利影响的可能解决方案是由用户在发现已有应用受影响后从设备主动移除新应用或禁止新应用的运行，从而确保负责重要功能的应用得以正常运行。
发明内容
为了避免同一设备上的应用部署之间的不利影响，已有的应用部署方案不仅要求用户在应用部署后实时关注应用的运行状况，而且很可能由于用户无法及时发现不利影响而造成不必要的损失。有鉴于此，在应用实际部署之前预先评估应用部署的影响将是有利的。本公开的实施例提供了这样用于预先评估应用部署的方案。
在本公开的第一方面中，提供了评估应用部署的方法。该方法包括响应于将第一应用部署到物理设备的请求，获得与第一应用相关的应用信息和与物理设备相关的设备信息，设备信息至少指示已部署在物理设备上的第二应用；基于应用信息和设备信息来创建模拟环境，模拟环境模拟物理设备在第一应用和第二应用被部署后的运行环境；以及通过在模拟环境中运行第一应用和第二应用来获取与第一应用相关联的评估结果，评估结果指示将第一应用部署到物理设备中对第二应用的影响。
通过本公开的实施例，通过在将应用实际部署之前自动评估该应用是否会影响设备中已部署的应用，能够主动且有效地避免有风险的应用部署，并且免除了对人工监测的要求。
在某些实施例中，获取与第一应用相关联的评估结果包括确定分别对应于第一应用和第二应用的第一测试用例和第二测试用例，第一测试用例和第二测试用例分别指定第一应用和第二应用的测试输入、运行条件和预期输出。基于第一测试用例和第二测试用例生成测试计划，测试计划指定第一测试用例和第二测试用例在模拟环境中的运行过程。在模拟环境中运行测试计划以获取评估结果。通过测试计划的自动生成，可以在模拟环境中测试第一应用和第二应用的不同输入、运行条件和输出，从而可能全面且可靠地评估第一应用的部署的影响。
在某些实施例中，第一测试用例和第二测试用例中的至少一个被预先创建和存储。这可以避免实时设计测试用例的耗时，并且同一应用对应的测试用例可以被重复利用。
在某些实施例中，物理设备是第一物理设备，并且其中创建模拟环境包括配置第二物理设备作为模拟环境，第二物理设备不同于第一物理设备。将单独的物理设备用于应用评估的目的可以避免应用评估对所关心的物理设备的影响。此外，单独的物理设备可以被重复用于为不同物理设备或不同应用进行部署评估。
在某些实施例中，配置第二物理设备包括将第一应用和第二应用部署到第二物理设备，基于应用信息和设备信息来配置第二物理设备。通过这种方式，第二物理设备在软件和硬件性能上均可以被配置为与第一 物理设备在第一应用和第二应用被部署后的情况类似，从而可以更准确、可靠地用于评估第一应用被部署后对第二应用的影响。
在某些实施例中，应用信息指示第一应用的标识和第一应用的资源要求。在某些实施例中，设备信息指示物理设备的可供应资源和可供应资源中的未占用资源，可供应资源包括计算资源、存储资源和通信资源中的至少一项。这样的应用信息和/或设备信息均可以有助于使模拟环境被创建为与物理设备上可能的真实环境相似。
在某些实施例中，获取与第一应用相关联的评估结果包括通过在模拟环境中运行第一应用和第二应用，确定第二应用是否会发生资源短缺或运行故障，以及响应于确定第二应用会发生资源短缺或运行故障，生成评估结果以指示在物理设备上部署第一应用会影响第二应用。第一应用对第二应用的影响可能体现在对物理设备上的资源的占用导致第二应用没有足够的运行资源或者影响第二应用的正常运行。通过在模拟环境中判断这种不期望的情况是否会发生，并通过评估结果来指示这种不期望的情况，可以有利地避免对第二应用的实际影响。
在某些实施例中，该方法还包括响应于评估结果指示将第一应用部署到物理设备中对第二应用无影响，使第一应用被部署到物理设备。在一个实施例中，使第一应用被部署到物理设备包括触发第一应用从应用服务器被下载到物理设备。由此，在确认第一应用的部署是安全的情况下，第一应用可以被真正部署到物理设备(例如从应用服务器被下载到物理设备)。
在某些实施例中，物理设备被包括在物联网(IoT)系统中。因此，可以为IoT系统中的设备提供有利的风险应用部署的评估，并且免除了对人工监测的要求。
在本公开的第二方面中，提供了电子设备。该电子设备包括处理器以及与处理器耦合的存储器，存储器具有存储于其中的指令，指令在被处理器执行时使电子设备执行动作。动作包括响应于将第一应用部署到物理设备的请求，获得与第一应用相关的应用信息和与物理设备相关的设备信息，设备信息至少指示已部署在物理设备上的第二应用；基于应 用信息和设备信息来创建模拟环境，模拟环境模拟物理设备在第一应用和第二应用被部署后的运行环境；以及通过在模拟环境中运行第一应用和第二应用来获取与第一应用相关联的评估结果，评估结果指示将第一应用部署到物理设备中对第二应用的影响。
在本公开的第三方面中，提供了用于评估应用部署的设备，包括用于执行根据第一方面的方法的各个实施例中的步骤的模块。
在本公开的第四方面中，提供了计算机程序产品，计算机程序产品被有形地存储在计算机可读介质上并且包括计算机可执行指令，计算机可执行指令在被执行时使至少一个处理器执行根据第一方面的方法的各个实施例。
在本公开的第五方面中，提供了计算机可读介质，其上存储有计算机可执行指令，计算机可执行指令在被执行时使至少一个处理器执行根据第一方面的方法的各个实施例。
提供发明内容部分是为了简化的形式来介绍对概念的选择，它们在下文的具体实施方式中将被进一步描述。发明内容部分无意标识本公开的关键特征或主要特征，也无意限制本公开的范围。
附图说明
下文将以明确易懂的方式通过对优选实施例的说明并结合附图来对本公开上述特性、技术特征、优点及其实现方式予以进一步说明，其中：
图1示出了用于常规应用部署的环境的示意图；
图2示出了根据本公开的一个实施例的应用部署的环境的示意图；
图3示出了根据本公开的一个实施例的评估应用部署的过程的流程图；
图4示出了根据本公开的另一个实施例的应用部署的环境的示意图；
图5A示出了根据本公开的一个实施例的用于模拟应用运行的过程的流程图；
图5B示出了根据本公开的一个实施例的用于评估结果生成和利用 的过程的流程图；
图6示出了根据本公开的又一个实施例的应用部署的环境的示意图；以及
图7示出了可以用来实施本公开的实施例的示例设备的框图。
具体实施方式
下面将参考附图中示出的若干示例实施例来描述本公开的原理。虽然附图中显示了本公开的优选实施例，但应当理解，描述这些实施例仅是为了使本领域技术人员能够更好地理解进而实现本公开，而并非以任何方式限制本公开的范围。
在本文中使用的术语“包括”及其变形表示开放性包括，即“包括但不限于”。除非特别申明，术语“或”表示“和/或”。术语“基于”表示“至少部分地基于”。术语“一个示例实施例”和“一个实施例”表示“至少一个示例实施例”。术语“另一实施例”表示“至少一个另外的实施例”。术语“第一”、“第二”等等可以指代不同的或相同的对象。下文还可能包括其他明确的和隐含的定义。
如以上提及的，用户可以请求应用服务器将应用部署到物理设备。图1示出了这样的常规应用部署的环境100。如图1所示，环境100包括应用服务器110和物理设备120。在图1的示例中，应用服务器110被部署在云130中。
云130可以是公共云或私有云，其被配置为集中管理和利用其中部署的软件和硬件资源，以用于为组织和/或个人提供诸如数据存储、共享、计算、分析等多种多样功能和/或服务。云130也可以被称为基于云的网络，其中的软件和/或硬件资源可以互连，并且还可以由外部设备和用户访问。基于云130的环境100可以实现用户、设备和产品的实时通信，构建高度灵活的个性化、数字化智能生态系统。
在云130中，应用服务器110被配置为管理各种应用。如本文所使用的，应用指的是被编写以用于实现特定功能的软件。当被安装在特定设备上时，应用可以被运行，从而实现相应功能，诸如数据采集控制、 数据处理、数学运算、流程管理、社交功能等等。应当理解，这里给出的仅仅是应用所提供的功能的一些非限制示例，并且根据需要，应用可以被设计用于实现各种各样的期望功能。
应用服务器110所管理的应用可以由多个来源(例如不同开发者、应用供应方等)提供。应用服务器110可以维护这些应用的安装文件和应用简档，并且控制这些应用的上传、合规性、下载、更新等等。因此，应用服务器110有时也可以被称为应用商店。用户102可以向应用服务器110请求，以请求将特定应用部署到物理设备120。用户102可以例如经由应用服务器110的输入设备(例如用户界面)或者经由用户终端设备向应用服务器110发送请求。在一些示例中，用户102也可以是物理设备120的用户。响应于来自用户102的请求，应用服务器110将对应的应用下载到物理设备120。物理设备120可以运行该应用，从而利用该应用来实现相应功能。
物理设备120可访问云130、特别是其中的应用服务器110。物理设备120可以是任何具有计算和通信能力的设备。作为非限制性示例，物理设备120可以是任意类型的固定设备、移动设备或便携式设备，包括台式计算机、膝上型计算机、笔记本计算机、上网本计算机、平板计算机、多媒体计算机、多媒体平板、互联网节点、移动手机、站点、物联网设备，等等。物理设备120可以是用户终端，或者可以是被布置在工业现场的设备，诸如用于电机、传感器等器件的控制设备、用于数据分析和计算的设备、用于作业指令控制和发送的设备等等。
在一些实现中，包括应用服务器110的云130以及与其相连的设备可以被称为物联网(IoT)系统。应当理解，出于描述的目的示出图1的环境。图1中示出的设备的布置和数目均是示意性的。在其他情况中，多个物理设备120可以连接和访问云130。应用服务器110也可以响应于来自多个用户的请求，用于将相同应用或不同的应用部署到不同的物理设备。虽然云130中仅示出包括应用服务器110，其他软件和/或硬件资源也可以被部署在云130中。本公开的范围在此方面不受限制。还应当理解，虽然在基于云的环境中进行描述，但本公开的实现实际上并不 局限于此，而是可以被实现在其他不同类型的环境中(例如，与基于云的网络无关的环境中)。
在传统应用部署中，用户请求的应用被直接部署到物理设备，例如在物理设备上安装和运行。然而，新部署的应用可能会影响设备上已部署的应用的正常运行。如果受影响的应用负责该设备的核心功能，这将是不期望的。虽然用户也可以在发现重要应用受影响时新部署的应用移除(例如卸载)或者禁止该应用的运行，然而如以上提及的，这不仅要求用户在应用部署后实时关注应用的运行状况，而且很可能由于无法及时发现应用的影响而导致不必要的损失。在很多情况下，对运行状态的实时关注可能难以实现，或者对应用的不利影响的发现可能会滞后。例如，在工业现场，要部署应用的设备数量较大，并且设备安装位置不支持的实时监测，这些原因都可能导致用户及时移除或禁用不期望的应用。
由此可见，直接的应用部署方案将会带来很大的潜在风险。因此，在应用实际部署之前预先评估应用部署的影响是有利的。根据本公开的实施例，提出了用于评估应用调度的方案。该方案要求在将请求的应用部署到物理设备之前，评估该应用对物理设备上的已部署应用的影响。这样的评估结果可以用于协助确定是否应当将请求的应用实际部署到物理设备，从而避免有风险的应用部署。
图2示出了根据本公开的一个实施例的应用部署的环境100的示意图。在图1所示的环境100的基础上，根据本公开的实施例，提供一个评估设备202，用于在应用服务器110接收到来自用户102的将某一应用部署到物理设备120的请求时，评估该应用对物理设备120上已部署的应用的影响。评估设备202被配置为与应用服务器110和物理设备120通信连接，以用于从应用服务器110和物理设备120接收信息，并且在一些情况下向应用服务器110发送信息。这样的通信连接可以是有线或无线连接。评估设备202可以包括具有通信和计算能力的设备或服务器。应当理解，虽然被示出为单独的设备，评估设备202可以被部署在云130中或者被集成到应用服务器110。
以下将参照图3来详细讨论评估设备202中的评估过程。图3示出 了根据本公开的一个实施例的用于评估应用部署的过程300的流程图。过程300可以由评估设备202来实现。
在框310，评估设备202获得与一个应用(为便于描述，称为第一应用)相关的应用信息和与物理设备120相关的设备信息。应用信息和设备信息的获取可以响应于将第一应用部署到物理设备120的请求。例如，应用服务器110接收到来自这样的请求(例如来自用户102的请求)之后，可以主动将第一应用的应用信息发送给评估设备202，并且向评估设备202指示期望评估第一应用对物理设备120上已部署的应用的影响。评估设备202接收来自应用服务器110的应用信息，并且向物理设备120请求和接收设备信息。
与第一应用相关的应用信息也可以被称为第一应用的应用简档，用于描述第一应用的一个或多个方面。在某些实施例中，第一应用相关的应用信息指示第一应用的标识。该标识可以用于在应用服务器110和物理设备120中用于区别第一应用。应用信息还可以指示第一应用的运行要求、诸如资源要求。通常在设备上运行时，应用将需要占用该设备上一定量的资源。在一些情况下，特定的资源量以及恰当的资源类型才能支持应用的正常运行。因此，应用信息可以指示第一应用对各类资源的类型和/或资源量的要求。
在某些实施例，第一应用的运行所涉及的资源可以包括计算资源、存储资源和通信资源中的一个或多个。计算资源诸如设备的处理器，包括中央处理单元(CPU)、诸如图形处理单元(GPU)、现场可编程门阵列(FPGA)之类的专用处理单元。存储资源包括设备的内存、闪存、持久性存储空间等。通信资源可以包括例如通用输入/输出(I/O)接口、总线等设备内部通信资源和/或诸如以太网端口等设备外部通信资源。
与物理设备120相关的设备信息描述物理设备120的一个或多个方面。设备信息至少指示已部署在物理设备120上的应用(为便于描述，称为第二应用)的标识。该标识可以用于在应用服务器110和物理设备120中用于区别第二应用。在某些实施例中，物理设备120可以部署有多个应用。在这种情况中，设备信息可以包括多个应用的标识。在另一 个实施例中，设备信息可以包括负责物理设备120上的关键任务的一个或多个应用的标识。例如，如果物理设备120被设置用于执行机器控制。为了实现这个任务，物理设备120上部署有用于传感器控制的应用和用于传感器数据分析的应用。设备信息中可以包括这两个应用的标识。
附加地，设备信息还可以指示物理设备120的资源状况，诸如物理设备120的可供应资源和未占用资源。可供应资源指的是物理设备120实际可支配的资源，包括计算资源、存储资源和通信资源中的一项或多项。未占用资源指的是实际可支配资源中未被占用的资源部分，因为物理设备120的一部分资源可能被已部署的应用和/或操作系统占用。
应当理解，以上仅给出了应用信息和设备信息的一些非限制性示例。在其他实施例中，应用信息和设备信息可以包括第一应用和物理设备的其他相关信息。例如，应用信息还可以包括第一应用对操作系统类型、网络类型等方面的要求。设备信息也可以包括物理设备的操作系统类型、设备类型、设备型号等方面的信息。
在框320，评估设备202基于获取的应用信息和设备信息来创建模拟环境。模拟环境模拟物理设备120在第一应用和第二应用被部署后的运行环境。为了避免影响物理设备120本身的运行环境，可以创建独立于物理设备120的实际运行环境的模拟环境，用于第一应用的评估。为了准确、可靠地评估第一应用对第二应用的影响，模拟环境可以被创建为与物理设备120在第一应用和第二应用被部署后的运行环境尽可能相似。例如，模拟环境在应用部署、硬件条件、网络条件等方面均与物理设备120的运行环境相似。
在某些实施例中，通过利用评估设备202可提供的资源(包括计算、存储和通信资源等)，模拟环境可以被创建在评估设备202。在另一些实施例中，评估设备202可以利用另一物理设备来创建模拟环境。为便于区分，物理设备120有时称为第一物理设备，而用于创建模拟环境的物理设备有时可以称为第二物理设备。第二物理设备不同于第一物理设备，以避免应用评估对第一物理设备的影响。此外，单独的第二物理设备还可以被重复用于为不同物理设备或不同应用进行部署评估。
图4示出了根据这样的实施例的环境100的一个示例。在图4示出的环境100中，提供物理设备410作为第二物理设备。物理设备410被配置为与评估设备202和应用服务器110通信连接。评估设备202基于应用信息和设备信息来配置物理设备410作为模拟环境。出于模拟的目的，物理设备410将被配置为在多个方面与物理设备120相同或相似，诸如在操作系统、存储和/或计算能力等方面相同或相似。在一个示例中，物理设备410甚至可以被选择为与物理设备120相同型号的设备。评估设备202可以基于应用信息和设备信息，确定物理设备410在应用部署、资源类型、可用资源量、操作系统类型等多方面的模拟要求。
具体地，评估设备202可以将第一应用和第二应用部署到物理设备410。例如，评估设备202可以将第一应用和第二应用的标识提供给物理设备410，并且触发物理设备410从应用服务器110下载第一应用和第二应用。在另一示例中，评估设备202也可以请求应用服务器110将第一应用和第二应用直接部署到物理设备410。评估设备202还基于应用信息和设备信息来配置第二物理设备，包括配置物理设备410在后续应用评估过程中可用的资源量和资源类型，以及第二物理设备向已部署的应用展现的操作系统类型、设备类型，等等。评估设备202可以将应用信息和设备信息转换为相应的配置参数并将这些参数提供给物理设备410。物理设备410可以基于接收到的参数来调整设备本身的配置，从而模拟物理设备120的运行环境。
继续参照图3，在框330，评估设备202通过在模拟环境中运行第一应用和第二应用来获取与第一应用相关联的评估结果。评估结果指示将第一应用部署到物理设备120中对第二应用的影响。通过模拟环境中运行第一应用和第二应用，即使第一应用会对第二应用造成影响，这样的运行也不会实际破坏物理设备120中的真实运行环境。
在某些实施例中，为了更好地测试第一应用和第二应用的不同输入、运行条件和输出，以全面且可靠地评估第一应用的部署的影响，可以基于测试用例来运行应用。图5A示出了根据本公开的另一个实施例的用于应用部署评估的过程502的流程图。过程502可以被认为是过程300 中获取评估结果的一个示例。过程502也可以由评估设备202来实现。
在框510，评估设备202确定分别对应于第一应用和第二应用的第一测试用例和第二测试用例。在物理设备410被配置为模拟环境的实施例中，评估设备202可以将确定的测试用例提供给物理设备410。在另外一些实施例中，评估设备202还可以充当控制装置，触发物理设备410确定第一应用和第二应用的测试用例。
第一测试用例和第二测试用例分别指定第一应用和第二应用的测试输入、运行条件和预期输出。在模拟环境中，第一应用和第二应用可能无法获得真实的应用输入，因此可以通过设计测试用例来给定测试输入以及相应的运行条件。根据这样的测试用例，在特定运行条件下，测试输入被提供给相应的应用，并记录应用运行后的输出。如果输出与预期输出相同，则认为应用运行正常；否则，则可以确定应用出错。测试用例的设计可以采用软件测试领域中任何当前已知或者将来待开发的技术来实现，本公开的范围在此方面不受限制。
与第一应用和/或第二应用的测试用例可以被预先创建和存储，例如存储在存储库中。因为应用服务器110提供的应用可以被部署到多个不同的物理设备，通过预先为这些应用创建和存储测试用例，可以避免实时设计测试用例的耗时，并且可以将预先存储的测试用例重复用于不同的应用部署评估过程。
图6示出了其中能够提供预先创建的测试用例的环境100的一个示例。在图6示出的环境100中，提供存储库610，用于存储预先创建的、与各个应用分别对应的测试用例。在图6的示例中，存储库610被示出为与物理设备410通信连接。虽然被图示为独立于云130，在其他实施例中，存储库610可以被部署在云130中，作为云130的一部分。物理设备410可以响应于评估设备202的触发而从存储库610获取第一应用和/或第二应用的测试用例。在某些实施例中，存储库610可以备选地或附加地被配置为与评估设备202通信连接。在这样的实施例中，可以由评估设备202从存储库610获得测试用例并且提供给设备410。
在某些实施例中，存储库610将测试用例和对应的应用的标识一起 存储。物理设备410或评估设备202可以基于第一应用和/或第二应用的标识从存储库610获取第一测试用例和/或第二测试用例。在某些实施例中，存储库610还可以包括更多其他应用对应的测试用例。
继续参照图5A，在框520，评估设备202基于第一测试用例和第二测试用例生成测试计划。测试计划指定第一测试用例和第二测试用例在模拟环境中的运行过程。例如，测试计划可以规定在整个运行过程中第一测试用例和第二测试用例的运行时间、操作范围、在输出错误情况下的处理等等。测试计划的生成也可以采用软件测试领域中任何当前已知或者将来待开发的技术来实现，本公开的范围在此方面不受限制。
在某些实施例中，除了提供可以与应用对应的测试用例之外，还可以提供额外的测试用例用于监测模拟环境(例如物理设备410)所模拟的操作系统、各类资源的可用性等等。这些测试用例也可以被用于生成测试计划。由此，可以实现对物理设备120的全面测试。
在框530，评估设备202在模拟环境中运行测试计划以获取评估结果。在配置物理设备410用作模拟环境的实施例中，评估设备202可以将所生成的测试计划提供给物理设备410，以由已安装第一应用和第二应用的物理设备410来运行测试计划，并从物理设备410获得测试计划的运行结果。在某些实施例中，评估设备202还可以充当控制装置，触发物理设备410生成以及运行测试计划。在测试计划的运行过程中，第一应用和第二应用根据各自的测试用例来运行。具体地，第一应用或第二应用在特定运行条件下被输入第一测试用例或第二测试用例中的测试输入，并且对应的输出将被记录以用于与测试用例中指定的预期输出进行比较。测试过程中的输出与预期输出不匹配，这意味着应用的运行结果不符合预期。
评估设备202可以基于测试计划的运行结果来确定与第一应用相关联的评估结果。这可以从运行结果的一个或多个方面来判断。特别地，基于运行结果判断第二应用的运行是否正常，从而判断第一应用在物理设备上部署第一应用是否会影响第二应用。在某些实施例中，如果评估设备202需要评估物理设备120上的两个或更多应用并且运行结果指示 其中的一个或多个应用受到影响，评估设备202生成相应的评估结果，以指示第一应用的部署对这些具体应用的影响。
图5B示出了在图5A的过程502之后基于测试计划后对评估结果的生成和利用的过程504。过程504也可以由评估设备202来实现。在框510，评估设备202通过确定第二应用是否会发生资源短缺或运行故障来确定评估结果。第一应用对第二应用的影响可能体现在对物理设备上的资源的占用导致第二应用没有足够的运行资源或者影响第二应用的正常运行。例如，第一应用可能会要求更多的计算资源或者占用更多的网络连接端口，从而导致第二应用无法占用更多资源。资源短缺可能会影响第二应用运行的效率，这是不期望的。在另一个示例中，第一应用的运行过程可能与第二应用的运行过程冲突，从而导致第二应用的运行发生故障。这些情况可以从测试计划的运行结果中确定。
如果在框540确定第二应用的资源要求均能得到满足并且运行结果均正确(例如与测试用例的预期输出相同或相似)，在框550，评估设备202生成评估结果以指示将第一应用部署到物理设备120中对第二应用无影响。然后，在框560，评估设备202使第一应用被部署到物理设备120。在某些实施例中，评估设备202可以触发第一应用从应用服务器被下载到物理设备。评估设备202可以向应用服务器110向应用服务器110发送指示无影响的评估结果或者关于应用部署的确认响应。响应于接收到的指示无影响的评估结果或者确认响应，应用服务器110将第一应用下载至物理设备120。由此，在确认第一应用的部署是安全的情况下，第一应用可以被真正部署到物理设备(例如从应用服务器被下载到物理设备)。
如果在框540确定第二应用会发生资源短缺或运行故障，在框570，评估设备202生成评估结果以指示在物理设备120上部署第一应用会影响第二应用。在这种情况下，在580，评估设备202使第一应用不被部署到物理设备120。在某些实施例中，评估设备202可以向应用服务器110发送无影响的评估结果或者关于应用部署的拒绝响应。响应于接收到这样的评估结果或响应，应用服务器110可以不向物理设备120部署 第一应用。此外，应用服务器110还可以向用户102提供第一应用无法被部署的指示，并且还可以向用户102提供无法被部署的原因是物理设备120中的第二应用将受到影响。关于受影响的第二应用的标识可以由评估设备202提供。
图7示出了可以用来实施本公开的实施例的示例设备700的示意性框图。设备700可以用于实现图3的过程300、图5A的过程502和/或图5B的过程502。设备700可以被实现为以上描述的评估设备202、物理设备120或物理设备410。
如图所示，设备700包括中央处理单元(CPU)701，其可以根据存储在只读存储器(ROM)702中的计算机程序指令或者从存储单元708加载到随机访问存储器(RAM)703中的计算机程序指令，来执行各种适当的动作和处理。在RAM 703中，还可存储设备700操作所需的各种程序和数据。CPU 701、ROM 702以及RAM 703通过总线704彼此相连。输入/输出(I/O)接口705也连接至总线704。
设备700中的多个部件连接至I/O接口705，包括：输入单元706，例如键盘、鼠标等；输出单元707，例如各种类型的显示器、扬声器等；存储单元708，例如磁盘、光盘等；以及通信单元709，例如网卡、调制解调器、无线通信收发机等。通信单元709允许设备700通过诸如因特网的计算机网络和/或各种电信网络与其他设备交换信息/数据。
处理单元701执行上文所描述的各个方法和处理，例如过程300、过程502和/或过程504。例如，在某些实施例中，过程300、过程502和/或过程504可被实现为计算机软件程序或计算机程序产品，其被有形地包含于计算机可读介质，诸如非瞬态计算机可读介质(例如存储单元708)。在某些实施例中，计算机程序的部分或者全部可以经由ROM 702和/或通信单元709而被载入和/或安装到设备700上。当计算机程序加载到RAM 703并由CPU 701执行时，可以执行上文描述的过程300、过程502和/或过程504的一个或多个步骤。备选地，在其他实施例中，CPU701可以通过其他任何适当的方式(例如，借助于固件)而被配置为执行过程300、过程502和/或过程504。
本领域的技术人员应当理解，上述本公开的方法的各个步骤可以通过通用的计算装置来实现，它们可以集中在单个的计算装置上，或者分布在多个计算装置所组成的网络上，可选地，它们可以用计算装置可执行的程序代码来实现，从而可以将它们存储在存储装置中由计算装置来执行，或者将它们分别制作成各个集成电路模块，或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。这样，本公开不限制于任何特定的硬件和软件结合。例如，本公开的某些实施例还包括各个程序模块和/或集成电路模块，用于执行过程300、过程502和/或过程504的一个或多个步骤和/或本公开的其他实施例中描述的一个或多个其他步骤。这些程序模块可以被包括或被体现在一个设备中，诸如图7的设备700中。
应当理解，尽管在上文的详细描述中提及了设备的若干装置或子装置，但是这种划分仅仅是示例性而非强制性的。实际上，根据本公开的实施例，上文描述的两个或更多装置的特征和功能可以在一个装置中具体化。反之，上文描述的一个装置的特征和功能可以进一步划分为由多个装置来具体化。
以上所述仅为本公开的可选实施例，并不用于限制本公开，对于本领域的技术人员来说，本公开可以有各种更改和变化。凡在本公开的精神和原则之内，所作的任何修改、等效替换、改进等，均应包含在本公开的保护范围之内。

Claims (25)

  1. 评估应用部署的方法，包括：
    响应于将第一应用部署到物理设备的请求，获得与所述第一应用相关的应用信息和与所述物理设备相关的设备信息，所述设备信息至少指示已部署在所述物理设备上的第二应用的标识；
    基于所述应用信息和所述设备信息来创建模拟环境，所述模拟环境模拟所述物理设备在所述第一应用和所述第二应用被部署后的运行环境；以及
    通过在所述模拟环境中运行所述第一应用和所述第二应用来获取与所述第一应用相关联的评估结果，所述评估结果指示将所述第一应用部署到所述物理设备中对所述第二应用的影响。
  2. 根据权利要求1所述的方法，其中获取与所述第一应用相关联的评估结果包括：
    确定分别对应于所述第一应用和所述第二应用的第一测试用例和第二测试用例，所述第一测试用例和所述第二测试用例分别指定所述第一应用和所述第二应用的测试输入、运行条件和预期输出；
    基于所述第一测试用例和所述第二测试用例生成测试计划，所述测试计划指定所述第一测试用例和所述第二测试用例在所述模拟环境中的运行过程；以及
    在所述模拟环境中运行所述测试计划以获取所述评估结果。
  3. 根据权利要求2所述的方法，其中所述第一测试用例和所述第二测试用例中的至少一个被预先创建和存储。
  4. 根据权利要求1所述的方法，其中所述物理设备是第一物理设备，并且其中创建所述模拟环境包括配置第二物理设备作为所述模拟环境，所述第二物理设备不同于所述第一物理设备。
  5. 根据权利要求4所述的方法，其中配置所述第二物理设备包括：
    将所述第一应用和所述第二应用部署到所述第二物理设备；以及
    基于所述应用信息和所述设备信息来配置所述第二物理设备。
  6. 根据权利要求1所述的方法，其中所述应用信息指示所述第一应用的标识和所述第一应用的资源要求。
  7. 根据权利要求1所述的方法，其中所述设备信息指示所述物理设备的可供应资源和所述可供应资源中的未占用资源，所述可供应资源包括计算资源、存储资源和通信资源中的至少一项。
  8. 根据权利要求1所述的方法，其中获取与所述第一应用相关联的评估结果包括：
    通过在所述模拟环境中运行所述第一应用和所述第二应用，确定所述第二应用是否会发生资源短缺或运行故障；以及
    响应于确定所述第二应用会发生资源短缺或运行故障，生成所述评估结果以指示在所述物理设备上部署所述第一应用会影响所述第二应用。
  9. 根据权利要求1所述的方法，进一步包括：
    响应于所述评估结果指示将所述第一应用部署到所述物理设备中对所述第二应用无影响，使所述第一应用被部署到所述物理设备。
  10. 根据权利要求9所述的方法，其中使所述第一应用被部署到所述物理设备包括：
    触发所述第一应用从应用服务器被下载到所述物理设备。
  11. 根据权利要求1所述的方法，其中所述物理设备被包括在物联网(IoT)系统中。
  12. 电子设备，包括：
    处理器；以及
    与所述处理器耦合的存储器，所述存储器具有存储于其中的指令，所述指令在被处理器执行时使所述电子设备执行动作，所述动作包括：
    响应于将第一应用部署到物理设备的请求，获得与所述第一应用相关的应用信息和与所述物理设备相关的设备信息，所述设备信息至少指示已部署在所述物理设备上的第二应用的标识；
    基于所述应用信息和所述设备信息来创建模拟环境，所述模拟环境模拟所述物理设备在所述第一应用和所述第二应用被部署后的运行 环境；以及
    通过在所述模拟环境中运行所述第一应用和所述第二应用来获取与所述第一应用相关联的评估结果，所述评估结果指示将所述第一应用部署到所述物理设备中对所述第二应用的影响。
  13. 根据权利要求12所述的电子设备，其中获取与所述第一应用相关联的评估结果包括：
    确定分别对应于所述第一应用和所述第二应用的第一测试用例和第二测试用例，所述第一测试用例和所述第二测试用例分别指定所述第一应用和所述第二应用的测试输入、运行条件和预期输出；
    基于所述第一测试用例和所述第二测试用例生成测试计划，所述测试计划指定所述第一测试用例和所述第二测试用例在所述模拟环境中的运行过程；以及
    在所述模拟环境中运行所述测试计划以获取所述评估结果。
  14. 根据权利要求13所述的电子设备，其中所述第一测试用例和所述第二测试用例中的至少一个被预先创建和存储。
  15. 根据权利要求12所述的电子设备，其中所述物理设备是第一物理设备，并且其中创建所述模拟环境包括配置第二物理设备作为所述模拟环境，所述第二物理设备不同于所述第一物理设备。
  16. 根据权利要求15所述的电子设备，其中配置所述第二物理设备包括：
    将所述第一应用和所述第二应用部署到所述第二物理设备；以及
    基于所述应用信息和所述设备信息来配置所述第二物理设备。
  17. 根据权利要求12所述的电子设备，其中所述应用信息指示所述第一应用的标识和所述第一应用的资源要求。
  18. 根据权利要求12所述的电子设备，其中所述设备信息指示所述物理设备的可供应资源和所述可供应资源中的未占用资源，所述可供应资源包括计算资源、存储资源和通信资源中的至少一项。
  19. 根据权利要求12所述的电子设备，其中获取与所述第一应用相关联的评估结果包括：
    通过在所述模拟环境中运行所述第一应用和所述第二应用，确定所述第二应用是否会发生资源短缺或运行故障；以及
    响应于确定所述第二应用会发生资源短缺或运行故障，生成所述评估结果以指示在所述物理设备上部署所述第一应用会影响所述第二应用。
  20. 根据权利要求12所述的电子设备，其中所述动作进一步包括：
    响应于所述评估结果指示将所述第一应用部署到所述物理设备中对所述第二应用无影响，使所述第一应用被部署到所述物理设备。
  21. 根据权利要求20所述的电子设备，其中使所述第一应用被部署到所述物理设备包括：
    触发所述第一应用从应用服务器被下载到所述物理设备。
  22. 根据权利要求12所述的电子设备，其中所述物理设备被包括在物联网(IoT)系统中。
  23. 用于评估应用部署的设备，包括用于执行根据权利要求1至11中任一项所述的方法的步骤的模块。
  24. 计算机程序产品，所述计算机程序产品被有形地存储在计算机可读介质上并且包括计算机可执行指令，所述计算机可执行指令在被执行时使至少一个处理器执行根据权利要求1至11中任一项所述的方法。
  25. 计算机可读介质，其上存储有计算机可执行指令，所述计算机可执行指令在被执行时使至少一个处理器执行根据权利要求1至11中任一项所述的方法。
PCT/CN2018/088029 2018-05-23 2018-05-23 用于评估应用部署的方法、设备、计算机程序产品和可读介质 WO2019222941A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
PCT/CN2018/088029 WO2019222941A1 (zh) 2018-05-23 2018-05-23 用于评估应用部署的方法、设备、计算机程序产品和可读介质
CN201880089377.5A CN111712795A (zh) 2018-05-23 2018-05-23 用于评估应用部署的方法、设备、计算机程序产品和可读介质
EP18919504.3A EP3754489A4 (en) 2018-05-23 2018-05-23 METHOD OF EVALUATING AN APPLICATION IMPLEMENTATION, DEVICE, COMPUTER PROGRAM PRODUCT, AND READABLE MEDIUM
US16/975,792 US20210248056A1 (en) 2018-05-23 2018-05-23 Method for evaluating application deployment, apparatus, computer program product, and readable medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/088029 WO2019222941A1 (zh) 2018-05-23 2018-05-23 用于评估应用部署的方法、设备、计算机程序产品和可读介质

Publications (1)

Publication Number Publication Date
WO2019222941A1 true WO2019222941A1 (zh) 2019-11-28

Family

ID=68616220

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/088029 WO2019222941A1 (zh) 2018-05-23 2018-05-23 用于评估应用部署的方法、设备、计算机程序产品和可读介质

Country Status (4)

Country Link
US (1) US20210248056A1 (zh)
EP (1) EP3754489A4 (zh)
CN (1) CN111712795A (zh)
WO (1) WO2019222941A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113742243A (zh) * 2021-09-17 2021-12-03 京东科技信息技术有限公司 应用评测方法、装置、电子设备和计算机可读介质

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11316757B1 (en) * 2020-06-23 2022-04-26 Amdocs Development Limited System, method, and computer program for consumer requirement based management for physical edge deployment of an application
CN114168446B (zh) * 2022-02-10 2022-07-22 浙江大学 一种面向移动端运行算法模型的模拟评测方法及设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102063299A (zh) * 2010-12-21 2011-05-18 东莞宇龙通信科技有限公司 移动终端中应用运行条件的评估方法及装置、移动终端
CN102331946A (zh) * 2011-09-28 2012-01-25 宇龙计算机通信科技(深圳)有限公司 应用程序的安装方法及系统
CN102779054A (zh) * 2012-06-15 2012-11-14 北京奇虎科技有限公司 应用程序的安装处理方法和装置、以及服务器
CN102902560A (zh) * 2012-09-13 2013-01-30 东莞宇龙通信科技有限公司 应用程序的下载方法及系统
CN105354042A (zh) * 2014-08-20 2016-02-24 中兴通讯股份有限公司 应用安装的处理方法及装置
CN105915601A (zh) * 2016-04-12 2016-08-31 广东欧珀移动通信有限公司 资源下载控制方法及终端

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101326515A (zh) * 2005-12-09 2008-12-17 微软公司 元数据驱动的应用程序部署
WO2008131460A2 (en) * 2007-04-24 2008-10-30 Stacksafe, Inc. System and method for creating a virtual assurance system
US8495008B2 (en) * 2008-10-14 2013-07-23 International Business Machines Corporation Application-aware recording and replay
US8667483B2 (en) * 2009-03-25 2014-03-04 Microsoft Corporation Device dependent on-demand compiling and deployment of mobile applications
US8479181B2 (en) * 2010-02-01 2013-07-02 International Business Machines Corporation Interactive capacity planning
CN103581247A (zh) * 2012-07-30 2014-02-12 杭州洱海科技有限公司 一种基于云计算环境的分布式Web测试方法
US10291472B2 (en) * 2015-07-29 2019-05-14 AppFormix, Inc. Assessment of operational states of a computing environment
US9760366B2 (en) * 2015-12-21 2017-09-12 Amazon Technologies, Inc. Maintaining deployment pipelines for a production computing service using live pipeline templates

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102063299A (zh) * 2010-12-21 2011-05-18 东莞宇龙通信科技有限公司 移动终端中应用运行条件的评估方法及装置、移动终端
CN102331946A (zh) * 2011-09-28 2012-01-25 宇龙计算机通信科技(深圳)有限公司 应用程序的安装方法及系统
CN102779054A (zh) * 2012-06-15 2012-11-14 北京奇虎科技有限公司 应用程序的安装处理方法和装置、以及服务器
CN102902560A (zh) * 2012-09-13 2013-01-30 东莞宇龙通信科技有限公司 应用程序的下载方法及系统
CN105354042A (zh) * 2014-08-20 2016-02-24 中兴通讯股份有限公司 应用安装的处理方法及装置
CN105915601A (zh) * 2016-04-12 2016-08-31 广东欧珀移动通信有限公司 资源下载控制方法及终端

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3754489A4 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113742243A (zh) * 2021-09-17 2021-12-03 京东科技信息技术有限公司 应用评测方法、装置、电子设备和计算机可读介质
CN113742243B (zh) * 2021-09-17 2024-03-01 京东科技信息技术有限公司 应用评测方法、装置、电子设备和计算机可读介质

Also Published As

Publication number Publication date
EP3754489A1 (en) 2020-12-23
US20210248056A1 (en) 2021-08-12
CN111712795A (zh) 2020-09-25
EP3754489A4 (en) 2021-09-29

Similar Documents

Publication Publication Date Title
US10678585B2 (en) Methods and apparatus to automatically configure monitoring of a virtual machine
US10001990B2 (en) Method and system for enhancing application container and host operating system security in a multi-tenant computing environment
US10162650B2 (en) Maintaining deployment pipelines for a production computing service using live pipeline templates
US10761873B2 (en) Evaluating distributed application performance in a new environment
US10860339B2 (en) Autonomous creation of new microservices and modification of existing microservices
CN107534571B (zh) 用于管理虚拟网络功能的方法、系统和计算机可读介质
US20170255460A1 (en) Analyzing deployment pipelines used to update production computing services using a live pipeline template process
US20170180459A1 (en) Building deployment pipelines for a production computing service using live pipeline templates
US10977167B2 (en) Application monitoring with a decoupled monitoring tool
US10594800B2 (en) Platform runtime abstraction
KR20170022028A (ko) 컨테이너 이미지 보안 검사 방법 및 그 장치
US10904128B2 (en) Testing functionality of an Internet of Things environment
GB2523338A (en) Testing a virtualised network function in a network
US20190258534A1 (en) Message oriented middleware with integrated rules engine
WO2019222941A1 (zh) 用于评估应用部署的方法、设备、计算机程序产品和可读介质
JP2020529066A (ja) クラウドサービスブローカーシステムにおいてオファーの機能を自動的に検証する技術
CN111831567B (zh) 应用的测试环境配置方法、装置、系统和介质
US10176067B1 (en) On-demand diagnostics in a virtual environment
US20130262845A1 (en) Method and system for generating a standardized system build file
CN108289114B (zh) 云端服务系统及处理云端服务异常的方法
US9730038B2 (en) Techniques to manage platform migrations
CN112269570B (zh) 安全代码开发方法及设备、计算设备和介质
US11968086B1 (en) Isolated environment provisioning in service mesh-based microservices systems
US11954469B2 (en) Bases for pattern-based cloud computing
CN118331679A (zh) 容器模拟测试系统、容器测试方法、计算机设备及产品

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18919504

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2018919504

Country of ref document: EP

Effective date: 20200917

NENP Non-entry into the national phase

Ref country code: DE