CN110727500A - Method, system, device and medium for integrating functional modules in system - Google Patents

Method, system, device and medium for integrating functional modules in system Download PDF

Info

Publication number
CN110727500A
CN110727500A CN201910921689.4A CN201910921689A CN110727500A CN 110727500 A CN110727500 A CN 110727500A CN 201910921689 A CN201910921689 A CN 201910921689A CN 110727500 A CN110727500 A CN 110727500A
Authority
CN
China
Prior art keywords
container
configuration file
containers
module
functional modules
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910921689.4A
Other languages
Chinese (zh)
Other versions
CN110727500B (en
Inventor
黄程
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Is According To Figure Network Technology Co Ltd
Original Assignee
Shanghai Is According To Figure Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Is According To Figure Network Technology Co Ltd filed Critical Shanghai Is According To Figure Network Technology Co Ltd
Priority to CN201910921689.4A priority Critical patent/CN110727500B/en
Publication of CN110727500A publication Critical patent/CN110727500A/en
Application granted granted Critical
Publication of CN110727500B publication Critical patent/CN110727500B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/48Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806Task transfer initiation or dispatching
    • G06F9/4843Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5022Mechanisms to release resources
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present application relates to a method, system, device and medium for integrating functional modules in a system. The integration method of the application can comprise the steps that a plurality of containers are built in the system, and the containers are strongly coupled with the system and used for guaranteeing the normal operation of functional modules in the system; setting a configuration file for each container, the configuration file comprising: the system resource occupancy rate of the container and the corresponding relation between the container and the functional module; and connecting the functional modules and the containers according to the configuration files, wherein each functional module is connected with one container, and the functional modules are weakly coupled with the containers. The container is arranged to be weakly coupled with the functional module, so that flexible management and flexible allocation of resources are conveniently performed on the functional module in the system.

Description

Method, system, device and medium for integrating functional modules in system
Technical Field
The present application relates to a method, system, device and medium for integrating functional modules in a system.
Background
The process is used as a basic unit for resource allocation and scheduling of the system, and the module integration is usually ensured through strong coupling between the process and the thread, and when the corresponding relationship between the thread and the process needs to be modified, the process is usually performed by directly stopping the system, which is not favorable for flexible application of the module. When a sub-module of a process is used by another process, the process cannot be released before the use is finished, and the system resources are always occupied until the module is completely used and can not be released. When only the sub-module of a certain process is started, all resources of the process also need to be occupied, which is not beneficial to flexible use of the resources.
Disclosure of Invention
A first aspect of the present application provides a method of integrating functional modules in a system, which may include,
establishing a plurality of containers in the system, wherein the containers are strongly coupled with the system and used for ensuring the normal operation of the functional modules in the system; setting a configuration file for each container, the configuration file comprising: the system resource occupancy rate of the container and the corresponding relation between the container and the functional module; and connecting the functional modules and the containers according to the configuration files, wherein each functional module is connected with one container, and the functional modules are weakly coupled with the containers.
The container is arranged to be weakly coupled with the functional module, so that flexible management and flexible allocation of resources are conveniently performed on the functional module in the system.
Further, the configuration file may be set according to the nature of the function modules in the system, each container connecting one or more function modules of the same nature. The property of the functional module may include at least one of failure rate, resource occupancy rate, function category, life cycle, usage frequency, engineering dependency, resource allocation priority, and other common engineering attributes of the functional module.
Further, the method may further include the step that the container polls the function module connected to the container at a preset polling period, and restarts the function module when the function module operates abnormally. The polling periods of different containers can be different, so that the normal operation of each functional module is ensured, and the total operation resource of the system is saved.
Further, the method may further include updating the configuration file of the container, and reconnecting the function module with the container according to the updated configuration file.
Further, connecting the function module and the container according to the configuration file may include determining whether the configuration file is correct, outputting error information and periodically checking the configuration file if the configuration file is incorrect, and connecting the function module and the container according to the correct configuration file until the correct configuration file is checked.
A second aspect of the present application provides a system, which may include a plurality of function modules for implementing functions of the system, and a plurality of containers for guaranteeing normal operation of the function modules in the system, the containers being strongly coupled with the system, and the containers being weakly coupled with the function modules; the container is configured to connect the function modules according to a configuration file of the container, and each function module is connected with one container, wherein the configuration file comprises the system resource occupancy rate of the container and the corresponding relation between the container and the function modules.
Further, the configuration file may be set according to the nature of the function modules in the system, each container connecting one or more function modules of the same nature. The property of the functional module may include at least one of failure rate, resource occupancy rate, function category, life cycle, usage frequency, engineering dependency, resource allocation priority, and other common engineering attributes of the functional module.
Further, the container may be further configured to poll the function module connected to itself at a preset polling period, and restart the function module when the function module operates abnormally. Wherein the polling periods may be different for different containers.
Further, the container may be further configured to reconnect the functional module according to the updated configuration file.
Further, the container may be further configured to determine whether the configuration file is correct when the container is connected to the function module, and in the case that the configuration file is incorrect, output error information and periodically check the configuration file until the correct configuration file is checked, and then connect the function module and the container according to the correct configuration file.
A third aspect of the present application provides an apparatus, which includes a processor and a memory, where the memory stores instructions, and the processor is configured to read the instructions in the memory to execute the method provided by the foregoing first aspect or any implementation manner of the first aspect.
A fourth aspect of the present application provides a machine-readable medium having stored thereon instructions which, when executed by a machine, cause the machine to perform the method provided by the foregoing first aspect or any implementation manner of the first aspect.
The system is provided with the container, the container is weakly coupled with the functional module, the relation between the container and the functional module can be changed by updating the configuration file in the running process of the system, the module integration can be ensured by utilizing the weak coupling relation, and meanwhile, the system has better robustness and easy modification, and is favorable for flexible application of resources.
Drawings
Fig. 1 is a schematic structural diagram of a module integration system according to an embodiment of the present application.
Fig. 2 is a flowchart of a method of integrating functional modules in a system according to an embodiment of the present application.
FIG. 3 is a flow diagram of a method of changing or updating a relationship between a container and a function module in a system according to an embodiment of the application.
Detailed Description
The present application is further described with reference to the following detailed description and the accompanying drawings. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting of the application. In addition, for convenience of description, only a part of structures or processes related to the present application, not all of them, is illustrated in the drawings.
Illustrative embodiments of the present disclosure include, but are not limited to, methods, systems, devices, and media for integration of functional modules in a system.
Various aspects of the illustrative embodiments will be described using terms commonly employed by those skilled in the art to convey the substance of their work to others skilled in the art. It will be apparent, however, to one skilled in the art that some alternative embodiments may be practiced using the features described in part. For purposes of explanation, specific numbers and configurations are set forth in order to provide a more thorough understanding of the illustrative embodiments. It will be apparent, however, to one skilled in the art that alternative embodiments may be practiced without the specific details. In some other instances, well-known features are omitted or simplified in order not to obscure the illustrative embodiments of the present application.
Moreover, various operations will be described as multiple operations separate from one another in a manner that is most helpful in understanding the illustrative embodiments; however, the order of description should not be construed as to imply that these operations are necessarily order dependent, and that many of the operations can be performed in parallel, concurrently, or simultaneously. In addition, the order of the operations may be re-arranged. The process may be terminated when the described operations are completed, but may have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, and the like.
References in the specification to "one embodiment," "an illustrative embodiment," etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may or may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Furthermore, when a particular feature is described in connection with a particular embodiment, the knowledge of one skilled in the art can affect such feature in combination with other embodiments, whether or not such embodiments are explicitly described.
The terms "comprising," "having," and "including" are synonymous, unless the context dictates otherwise. The phrase "A/B" means "A or B". The phrase "A and/or B" means "(A), (B) or (A and B)".
As used herein, the term module may refer to, be part of, or include an Application Specific Integrated Circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group) and/or a memory (shared, dedicated, or group). For executing one or more software or firmware programs, combinational logic circuits, and/or other suitable components that provide the described functionality.
In the drawings, some features of the structures or methods may be shown in a particular arrangement and/or order. However, it should be understood that such specific arrangement and/or ordering is not required. Rather, in some embodiments, these features may be described in a manner and/or order different from that shown in the illustrative figures. Additionally, the inclusion of structural or methodical features in a particular figure does not imply that all embodiments need to include such features, and in some embodiments, may not include such features or may be combined with other features.
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
FIG. 1 illustrates a block diagram of a module integration system according to some embodiments of the present application. In fig. 1 and the remaining figures, a letter following a reference number, such as "200 a," indicates a reference to an element having that particular reference number. A reference numeral without a subsequent letter in the text, such as "200", indicates a general reference to an implementation of the element bearing that reference numeral.
According to some embodiments of the present application, as shown in FIG. 1, a plurality of containers 200 (e.g., three containers shown: container I200a, container II200b, and container III 200c) may be coupled under main program 100 of system 10; each of the receptacles 200 may be coupled with one or more functional modules 300, for example, as shown in FIG. 1, receptacle I200a connects functional modules 300a and 300b, receptacle II200b connects functional modules 300c,300d, and 300e, and receptacle III 200c connects functional module 300 f. It should be noted that three containers and six functional modules are shown in fig. 1 merely as an example to illustrate the structure of the system 10, and not as a limitation of the present application, and according to some embodiments of the present application, more or fewer containers 200 may be included in the system 10, and each container 200 may also be connected to any one or more functional modules 300.
According to some embodiments of the present application, the container 200, as a modular structure strongly coupled to the system 10, may be used to ensure the normal operation of the function module 300 connected thereto, and at the same time, plays a role in scheduling and coordinating the function module 300. While the functional modules 300a, …,300f are weakly coupled to the containers 200a, 200b, and 200c, respectively, each for individually performing a particular program action or code behavior. Different functional modules 300 under the same container 200 share the resources of the container 200 and rely on the container 200 to ensure proper operation of itself.
Referring to fig. 1, each container 200 in the system 10 is configured with a corresponding configuration file, and each container 200 connects to a corresponding functional module 300 according to its own configuration file and ensures the normal operation of the corresponding functional module 300. According to some embodiments of the present application, the configuration file may include, but is not limited to: any one or more of system resource occupancy of the container, correspondence of the container to the function module, user data, function models, and the like.
According to some embodiments of the present application, the configuration file of the container in the system may be set according to the properties of the function module in the system, and the properties of the function module may include, but are not limited to, a failure rate, a resource occupancy rate, a function category, a life cycle, a usage frequency, an engineering dependency, a resource allocation priority, and/or the like of the function module. For example, according to some embodiments of the present application, two different containers may be provided for the functional module with the higher failure rate and the functional module with the lower failure rate, respectively, and each functional module with the higher failure rate is coupled to one container, while the functional module with the lower failure rate is coupled to another container, so as to prevent the functional module with the higher failure rate from affecting the normal operation of the functional module with the lower failure rate.
According to some embodiments of the present application, the container may poll the function module connected to itself at a preset polling period and handle an abnormal condition of the function module. For example, referring to fig. 1, the container II200b may poll the function modules 300c,300d, and 300e connected to itself at a preset polling period, and when one of the function modules 300c,300d, and 300e operates abnormally (e.g., abnormally sleeps or is shut down), the container II200b may restart the function module that operates abnormally.
In accordance with some embodiments of the present application, where multiple containers are included in a system, different containers may be polled at different polling periods. For example, in the system 10 shown in fig. 1, assuming that the function modules 300a and 300b connected to the container I200a are very easy to have abnormal operation, and the function modules 300c,300d, and 300e connected to the container II200b are relatively stable in operation, the polling period for polling the function modules 300c,300d, and 300e by the container II200b may be set to be longer than that of the container I200a, so as to ensure that the function modules operate normally, and at the same time, save the total computing resources of the system.
Taking a face recognition system as an example, in the face recognition system, a functional module (e.g., a camera module) implementing such a function similar to capturing a face may be very prone to malfunction due to the need for continuous capturing and processing data when the flow of people is large, and then a plurality of similar modules may be coupled to the container I and the container I may be configured to poll the functional modules connected thereto with a short polling period (e.g., 1 minute). While for some non-failing modules, e.g. main system modules, it is completely unnecessary and very wasteful of resources if they are also polled with a polling period of 1 minute, like such non-failing modules, they may all be coupled into container II and container II may be configured to poll the functional modules connected to it with a longer polling period, e.g. 10 days.
According to some embodiments of the present application, in the system 10, different amounts of resources may be allocated to the containers according to different requirements of different function modules, respectively, so as to prevent malignant competition between the function modules, for example, 200G resources may be allocated to the container I200a, 500G resources may be allocated to the container II200b, and 300G resources may be allocated to the container III 200 c. Then, the functional modules 300a and 300b under container I200a share the 200G resources of container I200a, while the functional modules 300c,300d, and 300e under container II200b share the 500G resources of container II200b, and the functional module 300f under container III 200c shares the 300G resources of container I200a alone. Therefore, for some important function modules needing to guarantee smooth operation, the problem that other modules cannot smoothly operate due to competition of the other modules for computing resources can be prevented.
For example, according to some embodiments of the present application, some functional modules may exist in the system, for example, security modules, which must be guaranteed to operate normally, in which case, a container may be separately provided for the security modules, so that the normal operation of the security modules can be guaranteed no matter how other functional modules in the system compete for resources.
For example, in an image processing system, a plurality of functional modules may be included, such as: the system comprises a feature extraction module, a video processing module, a database calling module, a distributed storage module, a retrieval module, a scene graph unloading module and the like. In case the system resources are not sufficient enough, several containers, e.g. three containers, may be provided in the system in order to prevent a vicious competition between the functional modules. Subsequently, each functional module is respectively coupled to the corresponding container, for example, the feature extraction module and the video processing module with higher GPU resource demand are coupled to the first container, and more GPU resources and less CPU resources are allocated to the first container, while the database call module, the distributed storage module and the retrieval module with higher CPU resource demand are coupled to the second container, and less GPU resources and more CPU resources are allocated to the second container, and for the remaining scene graph dump modules, since they need higher I/O (input/output) resources, they may be coupled to the third container, and higher I/O resources are allocated to them, the system may respectively configure 40%, 50%, 10% CPU resources, 100%, 0%, 0% GPU resources, and 10% for the first container, the second container and the third container, 30%, 60% of I/O resources. Then, the feature extraction module and the video processing module in the first container share 40% of CPU resources, 100% of GPU resources and 10% of I/O resources, the database calling module, the distributed storage module and the retrieval module in the second container share 50% of CPU resources, 0% of GPU resources and 30% of I/O resources, and the scene graph unloading module solely shares 10% of CPU resources, 0% of GPU resources and 60% of I/O resources.
The above image processing system is merely an example to illustrate the solution of the present application, and according to various embodiments of the present application, the solution provided by the present application can be applied to various systems including a plurality of functional modules, especially systems that need to modify the relationship between threads and processes from time to time or need to allocate resources.
In the system provided by the application, because the functional module and the container are in a weak coupling relationship, the functional module under the container can be started and killed very conveniently, and the corresponding relationship between the functional module and the container can be modified, and only the configuration file of the container needs to be modified, and the connection relationship between the functional module and the container is readjusted according to the modified configuration file.
For example, according to some embodiments of the present application, in the system 10 shown in fig. 1, assuming that the function module 300a is to be transferred from the container I200a to the container II200b, the configuration file of the container I200a may be updated, the function module corresponding to the container I200a may be updated from the function module 300a and the function module 300b to only the function module 300a, the configuration file of the container II200b may be updated, and the function module corresponding to the container II200b may be updated from the function module 300c, the function module 300d, and the function module 300e to: functional module 300b, functional module 300c, functional module 300d, and functional module 300 e. After the corresponding configuration file is updated, the container I and the container II poll the corresponding function module respectively, the container I decouples the function module 300b which is not in the configuration file after polling, and the container II establishes connection with the function module 300b which is not connected originally according to the configuration file.
According to other embodiments of the present application, assuming that the function module 300a in the system 10 is to be removed, the configuration file of the container I200a may also be updated, and the function module corresponding to the container I200a is updated from the function module 300a and the function module 300b to be the only function module 300 a. After the corresponding configuration file is updated, the container I polls to decouple the functional modules 300b that are not in the configuration file.
According to some embodiments of the present application, when a configuration file is updated or a function module and a container are connected according to the configuration file, whether the configuration file is correct or not may be determined, and if the configuration file is incorrect, error information may be output and the configuration file may be periodically checked until the correct configuration file is checked, and the function module and the container may be connected according to the correct configuration file. For example, in some cases, the container may not find the functional module indicated in the configuration file due to mismatching, at which point the configuration file may be checked back and modified or updated before reconnecting the functional module.
Fig. 2 illustrates a schematic diagram of a module integration method according to some embodiments of the present application. According to some embodiments of the application, the integration method may comprise the operations of:
step S101, a plurality of containers (e.g., container I200a, container II200b, and container III 200c in fig. 1) are established in the system, and the plurality of containers are strongly coupled to the system.
Step S102, setting a configuration file for each container according to the properties of the function modules in the system, wherein the configuration file may include, but is not limited to: any one or more of system resource occupancy of the container, correspondence of the container to the function module, user data, function models, and the like.
Subsequently, step S103, connecting the corresponding container and the functional module according to the configuration file of each container, wherein each functional module is connected with one container and the functional module is weakly coupled with the container. For example, as shown in FIG. 1, each of the receptacles 200 may be coupled with one or more functional modules 300, e.g., as shown in FIG. 1, receptacle I200a connects functional modules 300a and 300b, receptacle II200b connects functional modules 300c,300d, and 300e, and receptacle III 200c connects functional module 300 f.
According to some embodiments of the present application, the container 200, as a modular structure strongly coupled to the system 10, may be used to ensure the normal operation of the function module 300 connected thereto, and at the same time, plays a role in scheduling and coordinating the function module 300. While the functional modules 300a, …,300f are weakly coupled to the containers 200a, 200b, and 200c, respectively, each for individually performing a particular program action or code behavior. Different functional modules 300 under the same container 200 share the resources of the container 200 and rely on the container 200 to ensure proper operation of itself.
According to some embodiments of the present application, the configuration file of the container in the system may be set according to the properties of the function module in the system, and the properties of the function module may include, but are not limited to, a failure rate, a resource occupancy rate, a function category, a life cycle, a usage frequency, an engineering dependency, a resource allocation priority, and/or the like of the function module. For example, according to some embodiments of the present application, two different containers may be provided for the functional module with the higher failure rate and the functional module with the lower failure rate, respectively, and each functional module with the higher failure rate is coupled to one container, while the functional module with the lower failure rate is coupled to another container, so as to prevent the functional module with the higher failure rate from affecting the normal operation of the functional module with the lower failure rate.
After the connection is completed, in step S104, the container may poll the function module connected to the container in a preset polling period, and process the abnormal condition of the function module, so as to ensure the normal operation of the function module. For example, referring to fig. 1, the container II200b may poll the function modules 300c,300d, and 300e connected to itself at a preset polling period, and when one of the function modules 300c,300d, and 300e operates abnormally (e.g., abnormally sleeps or is shut down), the container II200b may restart the function module that operates abnormally.
In accordance with some embodiments of the present application, where multiple containers are included in a system, different containers may be polled at different polling periods. For example, in the system 10 shown in fig. 1, assuming that the function modules 300a and 300b connected to the container I200a are very easy to have abnormal operation, and the function modules 300c,300d, and 300e connected to the container II200b are relatively stable in operation, the polling period for polling the function modules 300c,300d, and 300e by the container II200b may be set to be longer than that of the container I200a, so as to ensure that the function modules operate normally, and at the same time, save the total computing resources of the system.
Taking a face recognition system as an example, in the face recognition system, a functional module (e.g., a camera module) implementing such a function similar to capturing a face may be very prone to malfunction due to the need for continuous capturing and processing data when the flow of people is large, and then a plurality of similar modules may be coupled to the container I and the container I may be configured to poll the functional modules connected thereto with a short polling period (e.g., 1 minute). While for some non-failing modules, e.g. main system modules, it is completely unnecessary and very wasteful of resources if they are also polled with a polling period of 1 minute, like such non-failing modules, they may all be coupled into container II and container II may be configured to poll the functional modules connected to it with a longer polling period, e.g. 10 days.
According to some embodiments of the present application, in operation S102 of setting a configuration file of a container, different resource occupation amounts may be respectively configured for the respective containers according to different requirements of different functional modules, so as to prevent malignant competition among the respective functional modules, for example, 200G resources may be allocated for container I200a, 500G resources may be allocated for container II200b, and 300G resources may be allocated for container III 200 c. Then, the functional modules 300a and 300b under container I200a share the 200G resources of container I200a, while the functional modules 300c,300d, and 300e under container II200b share the 500G resources of container II200b, and the functional module 300f under container III 200c shares the 300G resources of container I200a alone. Therefore, for some important function modules needing to guarantee smooth operation, the problem that other modules cannot smoothly operate due to competition of the other modules for computing resources can be prevented.
For example, according to some embodiments of the present application, some functional modules may exist in the system, for example, security modules, which must be guaranteed to operate normally, in which case, a container may be separately provided for the security modules, so that the normal operation of the security modules can be guaranteed no matter how other functional modules in the system compete for resources.
For example, in an image processing system, a plurality of functional modules may be included, such as: the system comprises a feature extraction module, a video processing module, a database calling module, a distributed storage module, a retrieval module, a scene graph unloading module and the like. In case the system resources are not sufficient enough, several containers, e.g. three containers, may be provided in the system in order to prevent a vicious competition between the functional modules. Subsequently, each functional module is respectively coupled to the corresponding container, for example, the feature extraction module and the video processing module with higher GPU resource demand are coupled to the first container, and more GPU resources and less CPU resources are allocated to the first container, while the database call module, the distributed storage module and the retrieval module with higher CPU resource demand are coupled to the second container, and less GPU resources and more CPU resources are allocated to the second container, and for the remaining scene graph dump modules, since they need higher I/O (input/output) resources, they may be coupled to the third container, and higher I/O resources are allocated to them, the system may respectively configure 40%, 50%, 10% CPU resources, 100%, 0%, 0% GPU resources, and 10% for the first container, the second container and the third container, 30%, 60% of I/O resources. Then, the feature extraction module and the video processing module in the first container share 40% of CPU resources, 100% of GPU resources and 10% of I/O resources, the database calling module, the distributed storage module and the retrieval module in the second container share 50% of CPU resources, 0% of GPU resources and 30% of I/O resources, and the scene graph unloading module solely shares 10% of CPU resources, 0% of GPU resources and 60% of I/O resources.
The above image processing system is merely an example to illustrate the solution of the present application, and according to various embodiments of the present application, the solution provided by the present application can be applied to various systems including a plurality of functional modules, especially systems that need to modify the relationship between threads and processes from time to time or need to allocate resources.
In the system provided by the application, because the functional module and the container are in a weak coupling relationship, the functional module under the container can be started and killed very conveniently, and the corresponding relationship between the functional module and the container can be modified, and only the configuration file of the container needs to be modified, and the connection relationship between the functional module and the container is readjusted according to the modified configuration file.
FIG. 3 illustrates a flow diagram of a method of changing or updating a relationship between a container and a function module in a system according to an embodiment of the application. As shown in fig. 3, according to some embodiments of the present application, the method may include the operations of:
in step S201, a list of containers in the current system is acquired.
Step S202, updating the configuration file of the container according to the requirement.
Taking the system 10 shown in FIG. 1 as an example, assume that a functional module 300a in the system 10 is to be transferred from container I200a to container II200b, according to some embodiments of the present application. Then, the configuration file of the container I200a may be updated, the function module corresponding to the container I200a is updated from the function module 300a and the function module 300b to only the function module 300a, the configuration file of the container II200b is updated, and the function module corresponding to the container II200b is updated from the function module 300c, the function module 300d, and the function module 300e to: functional module 300b, functional module 300c, functional module 300d, and functional module 300 e.
And S203-S207, enabling the container to poll the corresponding functional modules according to the updated configuration file, judging whether the container configuration is correct, and decoupling the functional modules which are not in the configuration file and connecting the functional modules which are not connected before according to the configuration file.
For example, in the above example, after the corresponding configuration file is updated, the container I and the container II respectively poll the corresponding function module, the container I decouples the function module 300b that is not in the configuration file after polling, and the container II establishes a connection with the function module 300b that is not connected originally according to the configuration file.
According to some embodiments of the present application, when performing a configuration file update operation or connecting a function module and a container according to a configuration file, a determination may be made as to whether the configuration file is correct, i.e., step S205, and in a case where the configuration file is incorrect (i.e., no determination is made in step S205), error information may be output and the configuration file may be periodically checked until the correct configuration file is checked, and then the function module and the container may be connected according to the correct configuration file, for example, the configuration file of the container may be updated after the error information is output, whether the updated configuration file is correct may be determined again, and in a case where the updated configuration file is correct, the function module and the container may be reconnected according to the updated configuration file. For example, in some cases, the container may not find the functional module indicated in the configuration file due to mismatching, at which point the configuration file may be checked back and modified or updated before reconnecting the functional module.
In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented in the form of instructions or programs carried on or stored on one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors or the like. When the instructions or program are executed by a machine, the machine may perform the integration methods described above. For example, the instructions may be distributed via a network or other computer readable medium. Thus, a machine-readable medium may include, but is not limited to, any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), such as floppy diskettes, optical disks, compact disc read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), Random Access Memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, or flash memory or tangible machine-readable memory for transmitting network information via electrical, optical, acoustical or other forms of signals (e.g., carrier waves, infrared signals, digital signals, etc.). Thus, a machine-readable medium includes any form of machine-readable medium suitable for storing or transmitting electronic instructions or machine (e.g., a computer) readable information.
There is also provided, in accordance with another embodiment of the present application, apparatus, including: a memory having instructions stored therein and a processor for reading the instructions in the memory to perform the method described above in connection with fig. 2 and 3.
The scheme provided by the application can realize more flexible application of resources under the condition that the system does not shut down. As long as the container module is ensured to survive, normal operation of the system can be ensured. And the relationship between the container module and the function module can be changed by updating the configuration file in the system, so that the system has better robustness and easy modification. The container is strongly coupled with the system and is weakly coupled with the function module at the same time, the setting and related content of the container module are relatively fixed, but the relation between the container and the function module is not fixed, and the change of the relation between the container and the function module can be carried out without stopping the system by modifying the configuration file, namely, the relation between the function module and the container can be adjusted at any time. In addition, in the scheme provided by the application, as the function modules are performed depending on the container, after all the function modules in the container module are stopped, the resources of the whole container can be released, so that the system resources are more fully utilized.
While the embodiments of the present application have been described in detail with reference to the accompanying drawings, the application of the present application is not limited to the various applications mentioned in the embodiments of the present application, and various structures and modifications can be easily implemented with reference to the present application to achieve various advantageous effects mentioned herein. Variations that do not depart from the gist of the disclosure are intended to be within the scope of the disclosure.

Claims (16)

1. A method for integrating functional modules in a system, comprising,
establishing a plurality of containers in the system, wherein the containers are strongly coupled with the system and used for ensuring the normal operation of the functional modules in the system;
setting a configuration file for each of the containers, the configuration file including: the system resource occupancy rate of the container and the corresponding relation between the container and the functional module; and
connecting the functional modules with the containers according to the configuration file, wherein each functional module is connected with one container, and the functional modules are weakly coupled with the containers.
2. The integration method according to claim 1, wherein the configuration file is set according to properties of the functional modules in the system, each container connecting one or more functional modules of the same property.
3. The integration method of claim 2, wherein the properties of the functional modules include at least one of failure rates, resource occupancy rates, function categories, life cycles, usage frequencies, engineering dependencies, and resource allocation priorities of the functional modules.
4. The integration method according to claim 1, further comprising the container polling the function module connected to itself at a preset polling period and restarting the function module when the function module is abnormally operated.
5. The integrated method of claim 4, wherein the polling periods of different containers are different.
6. The integration method of claim 1, further comprising,
and updating the configuration file of the container, and reconnecting the functional module and the container according to the updated configuration file.
7. The integration method according to claim 1, wherein the connecting the functional module and the container according to the configuration file comprises determining whether the configuration file is correct, outputting an error message and periodically checking the configuration file if the configuration file is incorrect, and connecting the functional module and the container according to the correct configuration file until the correct configuration file is checked.
8. A system, comprising,
a plurality of functional modules for implementing the functions of the system, and
a plurality of containers for securing normal operation of the functional modules in the system, the containers being strongly coupled to the system and the containers being weakly coupled to the functional modules;
wherein the container is configured to connect the function modules according to a configuration file of the container, and each function module is connected with one container, wherein the configuration file comprises the system resource occupancy rate of the container and the corresponding relation between the container and the function modules.
9. The system of claim 8, wherein the configuration file is configured according to properties of the functional modules in the system, each container connecting one or more functional modules of the same properties.
10. The system of claim 8, wherein the properties of the functional module include at least one of a failure rate, a resource occupancy, a function category, a lifecycle, a usage frequency, an engineering dependency, and a resource allocation priority of the functional module.
11. The system of claim 8, wherein the container is further configured to poll the function module connected to the container at a preset polling period, and restart the function module when the function module operates abnormally.
12. The system of claim 11, wherein the polling periods differ for different containers.
13. The system of claim 8, wherein the container is further configured to reconnect the functional module according to an updated configuration file.
14. The system of claim 8, wherein the container is further configured to determine whether the configuration file is correct when connecting with the function module, and in case of an error in the configuration file, output an error message and periodically check the configuration file until a correct configuration file is checked, and then connect the function module with the container according to the correct configuration file.
15. An apparatus, comprising:
a memory having instructions stored therein, an
A processor to read instructions in the memory to perform:
establishing a plurality of containers in a system, wherein the containers are strongly coupled with the system and used for ensuring the normal operation of functional modules in the system;
setting a configuration file for each of the containers, the configuration file including: the system resource occupancy rate of the container and the corresponding relation between the container and the functional module;
connecting the functional modules with the containers according to the configuration file, wherein each functional module is connected with one container, and the functional modules are weakly coupled with the containers.
16. A machine-readable medium having stored therein instructions which, when executed by a machine, the machine performs:
establishing a plurality of containers in a system, wherein the containers are strongly coupled with the system and used for ensuring the normal operation of functional modules in the system;
setting a configuration file for each of the containers, the configuration file including: the system resource occupancy rate of the container and the corresponding relation between the container and the functional module;
connecting the functional modules with the containers according to the configuration file, wherein each functional module is connected with one container, and the functional modules are weakly coupled with the containers.
CN201910921689.4A 2019-09-27 2019-09-27 Method, system, device and medium for integrating functional modules in system Active CN110727500B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910921689.4A CN110727500B (en) 2019-09-27 2019-09-27 Method, system, device and medium for integrating functional modules in system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910921689.4A CN110727500B (en) 2019-09-27 2019-09-27 Method, system, device and medium for integrating functional modules in system

Publications (2)

Publication Number Publication Date
CN110727500A true CN110727500A (en) 2020-01-24
CN110727500B CN110727500B (en) 2022-10-25

Family

ID=69218431

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910921689.4A Active CN110727500B (en) 2019-09-27 2019-09-27 Method, system, device and medium for integrating functional modules in system

Country Status (1)

Country Link
CN (1) CN110727500B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111427692A (en) * 2020-03-26 2020-07-17 中国平安财产保险股份有限公司 Function configuration method, device, electronic equipment and medium

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1959631A (en) * 2005-11-04 2007-05-09 上海启明软件股份有限公司 Self-contained technology for installing application software based on ITRON
CN102087599A (en) * 2011-02-24 2011-06-08 北京中电普华信息技术有限公司 J2EE-based Web application system development method and device
CN102413115A (en) * 2011-08-12 2012-04-11 乐视网信息技术(北京)股份有限公司 Client plug-in system
CN102681825A (en) * 2011-03-09 2012-09-19 北京航空航天大学 System and method for managing service run container
US20140040429A1 (en) * 2010-03-31 2014-02-06 Welch Allyn, Inc. Extensibility for Manipulation of Medical Data
CN104834541A (en) * 2015-03-30 2015-08-12 百度在线网络技术(北京)有限公司 Function module mounting method, mounting system and mounting system updating method
CN105183458A (en) * 2015-08-14 2015-12-23 深圳联友科技有限公司 Software platform development method and apparatus
CN106888233A (en) * 2015-12-15 2017-06-23 北京新媒传信科技有限公司 Data update system and method
US20170270031A1 (en) * 2016-03-15 2017-09-21 Fujitsu Limited Information processing apparatus, test execution method, and computer-readable recording medium
CN108108239A (en) * 2017-12-29 2018-06-01 咪咕文化科技有限公司 Method and device for providing service function and computer readable storage medium
US20190294779A1 (en) * 2018-03-23 2019-09-26 International Business Machines Corporation Secure system state extraction software extensibility via plugin sandboxing

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1959631A (en) * 2005-11-04 2007-05-09 上海启明软件股份有限公司 Self-contained technology for installing application software based on ITRON
US20140040429A1 (en) * 2010-03-31 2014-02-06 Welch Allyn, Inc. Extensibility for Manipulation of Medical Data
CN102087599A (en) * 2011-02-24 2011-06-08 北京中电普华信息技术有限公司 J2EE-based Web application system development method and device
CN102681825A (en) * 2011-03-09 2012-09-19 北京航空航天大学 System and method for managing service run container
CN102413115A (en) * 2011-08-12 2012-04-11 乐视网信息技术(北京)股份有限公司 Client plug-in system
CN104834541A (en) * 2015-03-30 2015-08-12 百度在线网络技术(北京)有限公司 Function module mounting method, mounting system and mounting system updating method
CN105183458A (en) * 2015-08-14 2015-12-23 深圳联友科技有限公司 Software platform development method and apparatus
CN106888233A (en) * 2015-12-15 2017-06-23 北京新媒传信科技有限公司 Data update system and method
US20170270031A1 (en) * 2016-03-15 2017-09-21 Fujitsu Limited Information processing apparatus, test execution method, and computer-readable recording medium
CN108108239A (en) * 2017-12-29 2018-06-01 咪咕文化科技有限公司 Method and device for providing service function and computer readable storage medium
US20190294779A1 (en) * 2018-03-23 2019-09-26 International Business Machines Corporation Secure system state extraction software extensibility via plugin sandboxing

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
ANDREW J. YOUNGE等: "A Tale of Two Systems: Using Containers to Deploy HPC Applications on Supercomputers and Clouds", 《2017 IEEE INTERNATIONAL CONFERENCE ON CLOUD COMPUTING TECHNOLOGY AND SCIENCE (CLOUDCOM)》 *
宋杰等: "面向代码的软件能耗优化研究进展", 《计算机学报》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111427692A (en) * 2020-03-26 2020-07-17 中国平安财产保险股份有限公司 Function configuration method, device, electronic equipment and medium

Also Published As

Publication number Publication date
CN110727500B (en) 2022-10-25

Similar Documents

Publication Publication Date Title
CN106874031B (en) Method and device for starting system program of terminal equipment
CN108462760B (en) Electronic device, automatic cluster access domain name generation method and storage medium
CN109558234A (en) A kind of timed task dispatching method and device
CN111209110B (en) Task scheduling management method, system and storage medium for realizing load balancing
CN109766172B (en) Asynchronous task scheduling method and device
CN109981737B (en) Network request processing method, device, terminal and storage medium
US20100122263A1 (en) Method and device for managing the use of a processor by several applications, corresponding computer program and storage means
CN112486642B (en) Resource scheduling method, device, electronic equipment and computer readable storage medium
CN110727500B (en) Method, system, device and medium for integrating functional modules in system
CN112860387A (en) Distributed task scheduling method and device, computer equipment and storage medium
CN109561134B (en) Electronic device, distributed cluster service distribution method and storage medium
CN114629960A (en) Resource scheduling method, device, system, device, medium, and program product
CN111538585A (en) Js-based server process scheduling method, system and device
CN113703975A (en) Model distribution method and device, electronic equipment and computer readable storage medium
CN113608896B (en) Method, system, medium and terminal for dynamically switching data streams
CN113806055A (en) Lightweight task scheduling method, system, device and storage medium
CN110058866B (en) Cluster component installation method and device
CN110134443A (en) Method and apparatus for executing option ROM of accessory in computing device
CN109284141B (en) Plug-in management system and method
CN115408106A (en) Method and system for sharing algorithm resources among virtual machines
CN114077493A (en) Resource allocation method and related equipment
CN112506626A (en) Application program starting method, computer equipment and storage medium
CN112188143A (en) Method and device for releasing resources
CN111726418B (en) Cloud resource allocation method and device, terminal equipment and storage medium
CN117539626A (en) Container resource allocation method, device, system and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant