WO2021043124A1 - 一种kbroker分布式操作系统、存储介质和电子设备 - Google Patents

一种kbroker分布式操作系统、存储介质和电子设备 Download PDF

Info

Publication number
WO2021043124A1
WO2021043124A1 PCT/CN2020/112816 CN2020112816W WO2021043124A1 WO 2021043124 A1 WO2021043124 A1 WO 2021043124A1 CN 2020112816 W CN2020112816 W CN 2020112816W WO 2021043124 A1 WO2021043124 A1 WO 2021043124A1
Authority
WO
WIPO (PCT)
Prior art keywords
app
module
kbroker
server
application
Prior art date
Application number
PCT/CN2020/112816
Other languages
English (en)
French (fr)
Inventor
程延辉
程耀辉
Original Assignee
程延辉
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 程延辉 filed Critical 程延辉
Publication of WO2021043124A1 publication Critical patent/WO2021043124A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1479—Generic software techniques for error detection or fault masking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/60—Software deployment
    • G06F8/61—Installation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/60—Software deployment
    • G06F8/65—Updates

Definitions

  • the invention relates to the field of server-side software development and operation and maintenance systems, in particular to a kbroker distributed operating system, storage medium and electronic equipment.
  • the main technical solutions include microservices, message queues, and timing programs. Under normal circumstances, a combination of three programs is required to obtain better results. For server operation and maintenance, the more advanced is the docker technology based on kubernetes management.
  • Microservice is to split the entire large function into multiple relatively independent small functions.
  • the microservice also needs to consider the large load problem caused by the increase in the amount of visits during the programming. From this perspective, the microservice is still a To cope with the problem of large load and high concurrency, it is only reduced from a complex large function to a relatively simple small function.
  • the simple function separation also increases the complexity of calling between different microservices and the overall system operation of the microservices. Difficulty of governance.
  • the message queue is to cope with peak shaving, asynchronous message processing and decoupling between realization functions during momentary high load.
  • the message queue has open source programs that can be used, the writing of business logic code needs to split the original logic into messages.
  • the two logics of the producer and the executor of the message, additional deployment procedures and code splitting will lead to large-scale use, development and operation and maintenance are not an easy task.
  • the timing program is to solve the tasks that are executed regularly without external access triggering. Normally, the delay tasks that the program needs to use will also be triggered at the corresponding time point in the way of timing tasks. Timed tasks need to be configured on the server side. In addition, special code is needed to save the tasks to be processed and the specific processing of the callback function when the task is triggered. If there are many different timed tasks, it is very good for development and operation and maintenance. Troublesome and error-prone.
  • Docker technology based on kubernetes can be simply understood as a simplified virtual machine solution. Docker is more lightweight than virtual machines. It is mainly to solve the problems of program deployment and operation, especially the differences in program operating environments. The operation of this operation and maintenance system requires relatively high abilities of operation and maintenance personnel, and it also requires certain adaptation development from the code level. It is still very challenging to play with this system. Even if we consider it as a simpler cloud service (such as Facebook Cloud) plus automated deployment, it still requires strong operation and maintenance capabilities.
  • the purpose of the present invention is to provide a kbroker distributed operating system and storage medium to solve the problems in the prior art:
  • microservices are used to split functions, message queues are used to decouple functions, and time delay programs are added to complete the development.
  • This development model uses many framework components and the system is complex. The developer's requirements are high;
  • a kbroker distributed operating system including:
  • the kbroker_server module is used to manage the server where it is located in the distributed operating system.
  • the kbroker_server module manages the app_service module, app_allocator module, and storage resources on the server where it is located; data communication connections between multiple kbroker_server modules;
  • the business layer module is used to implement the business logic of the entire distributed operating system; the business logic is split into several applications; each application includes a corresponding set of app_allocator modules and at least one set of app_service modules, app_service
  • the module is used to run the business logic, and the app_allocator module is used to manage the running resources of the application.
  • the running resources are server resources required by the application to execute its business logic; the app_allocator module distributes and runs based on the demand for the running resources
  • the app_service module of the application program; the business logic is implemented by the logical processing of the app_object object of the business layer running on the app_service module and the mutual calls between the app_object objects;
  • the kbroker_super module is used to manage the server where the kbroker_server module is located through the kbroker_server module; all the program processes in the distributed operating system are managed through the kbroker_server module, and the process number program_id is set for each program process; the kbroker_server module manages all the servers on the distributed system Allocate the used and available storage resources, and set the resource number resource_id for each allocated storage resource and save the corresponding relationship between the resource number resource_id and the kbroker_server module; manage business layer applications, and for each business layer Application setting application number app_id;
  • the gateway module is a dedicated business layer module for receiving and responding to external requests.
  • application programs in the distributed operating system include:
  • Storage applications whose business logic involves the use of storage resources to store data.
  • the storage resource is a storage medium used to save data to prevent data loss after the kbroker distributed operating system restarts
  • the kbroker_server module manages the specific generation of storage resources, including a local file used to save the connection and access methods of storage resources after generation.
  • the kbroker_server module passes it in when the program process is started.
  • the connection access method of storage resources allows the program process to connect and access the storage resources it needs; the kbroker_server module is also used to manage the destruction and deletion of storage resources.
  • the kbroker_super module includes a storage resource for storing server information, the correspondence between resource ID resource_id and the kbroker_server module, application information, and storage resources used by the kbroker_super module.
  • the kbroker_super module maintains and records the available status of the CPU, memory, disk, and network on the server where the kbroker_server module is located, and selects the kbroker_server module for newly allocated storage resources and newly started program processes based on this.
  • the program processes in the kbroker distributed operating system communicate through the process number program_id; the kbroker_server module stores the association relationship between the process number program_id and the kbroker_server module on the server where it is located, and the message transmission in the system is forwarded by the kbroker_server module.
  • the kbroker_super module manages its server through the kbroker_server module:
  • the kbroker_super module realizes adding and deleting the server where the kbroker_server module is located in the kbroker distributed operating system by starting and closing the kbroker_server module on the server;
  • Each server used in the kbroker distributed operating system includes a unique kbroker_server module
  • the kbroker_server module monitors the operating status of the server where it is located and the availability of CPU, memory, disk, and network, and reports to the kbroker_super module;
  • the kbroker_server module monitors the connection status and reports it to the kbroker_super module.
  • the kbroker_super module obtains and handles the abnormal operation of the kbroker_server module accordingly.
  • the kbroker_super module realizes adding and deleting the server where the kbroker_server module is located in the kbroker distributed operating system by starting and closing the kbroker_server module on the server, which specifically includes:
  • the kbroker_super module saves the login information of all servers, the number of the server, and the execution command to start the kbroker_server module;
  • the kbroker_super module uses the server login information to log in to the server;
  • the kbroker_super module executes the startup command on the logged-in server to start the kbroker_server module;
  • the kbroker_server module After the startup, the kbroker_server module connects to the kbroker_super module according to the information of the kbroker_super module in the startup command, and requests registration verification from the kbroker_super module;
  • the kbroker_super module verifies the registration request. If the verification is successful, the kbroker_server module starts successfully;
  • the kbroker_super module deletes its server from the distributed operating system by notifying the kbroker_server module that has been successfully verified to shut down by itself.
  • the kbroker_super module is also used to add a new server to the distributed operating system during operation:
  • the kbroker_super module numbers the new server
  • the kbroker_super module starts and verifies the kbroker_server module corresponding to the new server. If the verification is successful, the new server is successfully added, and the kbroker_super module saves the information of the new server.
  • the kbroker_super module selects the kbroker_server module for each app_allocator module of the application according to the number of app_allocator modules required by the application and starts the app_allocator module through the kbroker_server module;
  • the kbroker_super module selects the main app_allocator module corresponding to the application from the app_allocator module of the started application, and sets the other started app_allocator modules of the application to the slave app_allocator module and completes synchronization with the main app_allocator module to realize the main Slave disaster recovery, and maintain master-slave disaster recovery during operation;
  • the main app_allocator module starts the app_service module, and adds or reduces the running number of app_service modules according to the needs of running resources during the running process.
  • the kbroker_super module saves the relationship between the resource number resource_id of the storage resource used by the app_allocator module and the application number app_id of the application to which the app_allocator module belongs;
  • the kbroker_super module starts an application that uses storage resources in its app_allocator module, it finds the resource number resource_id of all storage resources associated with the application number app_id of the application, and finds the corresponding kbroker_server module according to each resource number resource_id and starts it The app_allocator module that uses this storage resource. If there is no app_allocator module to start successfully, the application fails to start; if the app_allocator module starts successfully, then judge the number of successful startups, and if the number meets the needs of master-slave disaster recovery, complete and maintain the master-slave Disaster recovery, otherwise, first start the new app_allocator module to meet the number of master-slave disaster recovery, and then complete and maintain the master-slave disaster recovery;
  • the kbroker_super module When the kbroker_super module starts a new app_allocator module, a storage resource is created, and the new app_allocator module is started through this storage resource.
  • the kbroker_super module saves the resource number resource_id of the newly created storage resource and the application number of the application to which the app_allocator module belongs The relationship of app_id;
  • app_service module is managed by the app_allocator module corresponding to its application:
  • the kbroker_server module is selected through the kbroker_super module, and the selected kbroker_server module completes the specific startup work;
  • the app_allocator module deletes or modifies the routing information cache of the app_object object running on the app_service module, and decides whether to start a new app_service module as needed;
  • the app_allocator module When there are too many app_service modules, the app_allocator module will close the idle app_service modules.
  • the kbroker_server module is responsible for managing the app_allocator module and app_service module running on the server where it is located:
  • the app_allcator module and app_service module are started by the kbroker_server module. After startup, they run on the same server as the kbroker_server module and connect to the kbroker_server module through system inter-process communication;
  • the kbroker_server module monitors the abnormal conditions of the app_allocator module and app_service module, and reports to the kbroker_super module for processing;
  • the kbroker_server module performs the management operations of overload setting, migration, and shutdown of the app_allocator module and app_service module running on it according to the overall usage of the server's CPU, memory, disk, and network.
  • app_allocator module is also used to create app_object objects for management applications:
  • the mode of app_object object of computing application includes permanent existence mode and temporary existence mode, and the mode of app_object object of storage application is permanent existence mode;
  • the app_allocator module creates and starts the app_object object by executing the app_service module;
  • the app_allocator module saves the routing information of the app_object object, the routing information is which app_service module the app_object object is running on, and provides an interface for requesting routing information to the outside;
  • the app_allocator module provides corresponding interface functions to create and delete app_object objects in permanent existence mode
  • the app_object object in the permanent existence mode starts running when it is accessed. If there is no access request for a long time, the app_object object will be closed and exited from running;
  • the app_allocator module that manages the app_object object in the permanent existence mode, it includes a storage resource for saving the existing app_object object;
  • app_object object of temporary existence mode For the app_object object of temporary existence mode, it is allocated by the app_allocator module according to its needs;
  • the app_allocator module meets the specific requirements of the app_object module by the application by adding exclusive functions.
  • app_allocator module of the storage application maintains the master-slave disaster recovery of the app_service module:
  • the app_allocator module of the storage application compiles several corresponding app_service modules into a group, and sets the group number virtual_id for each group to maintain the master-slave disaster recovery of the app_service module in each group;
  • the app_service modules of storage applications have corresponding storage resources.
  • the app_allocator module of storage applications saves the association between the group number virtual_id of the group and the resource number resource_id of the storage resource used by the app_service module in the group. ;
  • the existing app_object object is bound to the group one-to-one, and the app_allocator module of the storage application saves the correspondence; a new app_object object selects a group to bind when applying for creation, and creates a new one when there is no available group Group and the app_service module in the group, and then select a group to bind;
  • the app_allocator module of the storage application processes the routing information request for the app_object object, after finding the group number virtual_id of the group corresponding to the app_object object, it finds the process number program_id of the main app_service module in the group corresponding to the group number virtual_id and returns to the request square.
  • app_service module runs the specific logic of the app_object object:
  • the app_object object is uniquely identified by the application number app_id of the application and the object number object_id of the app_object object on the application.
  • the business layer uses the application number app_id of the application and the object number object_id of the app_object object on the application to identify the app_object object. access;
  • the business layer implements business logic by rewriting the app_object_i virtual base class provided by the app_service module;
  • the logical processing of a single app_object object can only run on a single app_service module, and a single app_service module can run multiple app_object objects at the same time;
  • the kbroker distributed operating system provides a virtual interface for creating an app_object object through the object number object_id, the app_service module creates an app_object object through the virtual interface rewritten by the business layer, and initializes the app_object object through the initialization interface of the app_object object;
  • the business logic of the same app_object object is queued for processing on the same thread on the app_service module.
  • Each task event of the app_object object is encapsulated into a coroutine for execution.
  • the kbroker distributed operating system provides coroutine support for blocking interfaces ;
  • the app_service module caches the routing information of the app_object object, obtains the routing information through the app_allocator module corresponding to the app_object object to update the cache, and updates the cache of the source app_object object routing information in the access request when other app_object objects are accessed;
  • the external access of the business layer is based on the app_object object, and the system provides an interface for accessing the external app_object object, which is implemented by the app_service module; the app_service module finds the process number program_id of the app_service module where the app_object object is running according to the cache or routing information requested from the corresponding app_allocator module, Send the message whose destination is the process number program_id so that the message is sent to the app_service module where the app_object object is running;
  • the app_service module provides coroutine support for remote function calls that access external app_object objects
  • the app_service module After the app_service module receives the pending message of the app_object object, it checks the operating status of the target app_object object and processes it;
  • the app_service module informs the app_allocator module that it will no longer allocate new app_object objects to run on it when it monitors that it is overloaded;
  • the app_service module provides an interface for the business layer to add asynchronous events and delayed events, coroutine locks and coroutine signals;
  • the app_service module supports migrating the running app_object object to other app_service modules in the same application
  • the app_service module of the storage application provides a master-slave disaster recovery framework for the app_object object, and cooperates with the master-slave synchronization-related interface in the app_object_i virtual base class rewritten by the business layer to realize the master-slave disaster recovery function;
  • the app_service module of a storage application provides an interface for transferring data between master and slave app_object objects.
  • business layer implements business logic by rewriting the app_object_i virtual base class:
  • the app_object_i virtual base class of computing applications and the app_object_i virtual base class of storage applications include function_api interface, notify_api interface, initialization interface, delete interface, export data interface, and import data interface; function_api interface is used to process the return value Access requests, the notify_api interface is used to process access requests that do not require a return value;
  • the business layer module implements business logic processing by rewriting the function_api interface and notify_api interface; the business layer module rewrites the initialization interface for the app_service module to perform initialization after creating the app_object object; the business layer module rewrites the delete interface for the app_service module when the app_object object is closed Release the data created by the business layer; the export data interface and the import data interface are used for the migration of app_object objects. By default, app_object object migration is not supported. If the business layer rewrites these two interfaces, the application supports the migration of app_object objects. When the business load is high, migrate the app_object object to the app_service module with low load;
  • the app_service module of storage applications implements master-slave disaster recovery, and its app_service module implements the overall export and import interface and master-slave switching interface for the data of its storage-type resources.
  • its app_object_i virtual base class also includes a master-slave synchronization interface, master-slave The synchronization interface cooperates with the master-slave disaster recovery framework of the app_service module to realize the master-slave disaster recovery of the logical data of the business layer.
  • the gateway module includes a session_allocator module and a kbroker_gateway module;
  • the session_allocator module is extended and developed based on the basic functions of the app_allocator module, saves the socket monitoring information of the kbroker_gateway module, and realizes load balancing of multiple kbroker_gateway modules;
  • the kbroker_gateway module is extended and developed based on the basic functions of the app_service module, and supports various network-based external request access methods, and the app_object object running on it is an external request connection that is passively created and includes the object number object_id;
  • the kbroker_gateway module is used to receive an external request, and parse the request from the request to obtain the application number app_id and the object number object_id corresponding to the target app_object object;
  • the kbroker_gateway module supports setting a default message processing strategy for external requests and a special processing strategy based on the application number app_id of the target app_object object corresponding to the request.
  • the processing strategy is determined by the application number app_id of the target app_object object parsed from the external request. ; If there is a special processing strategy, the special processing strategy is executed, otherwise the default processing strategy is executed;
  • the kbroker_gateway module also includes the notify_api interface and function_api interface that are provided for other applications in the system to access; the notify_api interface and the function_api interface constitute the message processing strategy in the system, and the message processing strategy in the system is the default message processing strategy or based on
  • the special message processing strategy of the application number app_id of the message source app_object object supports the setting of a special message processing strategy for a certain application number app_id.
  • the kbroker distributed operating system includes a singleton application:
  • the singleton application program manages the singleton type business logic in the kbroker distributed operating system, and the singleton type business logic is the business logic that cannot be distributed to multiple places but is only executed in one program process;
  • the singleton application is a special application in the kbroker distributed system; the application number of the singleton application is the singleton application number singleton_app_id specially assigned by the system; the app_service module of the singleton application is The program process of various singleton business logics with different functions;
  • the app_allocator module of the singleton application assigns a singleton_object_id to each app_service module, which is used to present the app_service module as an object of the singleton application relative to other applications.
  • the business layer uses the singleton application number singleton_app_id And the singleton_object_id of the app_service module to access the app_service module of the singleton application;
  • the app_service module of the singleton application supports master-slave disaster recovery, and the app_allocator module manages and maintains the master-slave disaster recovery of the app_service module;
  • the app_service module of the singleton application includes a unique business layer app_object object, and all business layer requests are processed by this app_object object;
  • the app_allocator module of the singleton application is provided by the kbroker distributed operating system, and the kbroker_super module and the kbroker_server module provide support for starting the singleton application.
  • the kbroker_super module supports master-slave disaster recovery.
  • the master kbroker_super module starts a new slave kbroker_super module to implement replacement;
  • all slave kbroker_super modules elect to determine one of them to switch from the kbroker_super module to the master kbroker_super module, and then start a new replacement from the kbroker_super module.
  • the management background provides an access interface to the kbroker distributed operating system.
  • the kbroker distributed operating system obtains the access interface at startup through configuration or startup parameters; the kbroker distributed operating system reports status information through the interface, and the status information includes the connection information of the management gateway , Monitoring information of server operation, operating status of various applications, alarms for abnormal conditions, and early warnings for specific conditions;
  • the management background obtains the operating information of the kbroker distributed operating system through the management gateway and sends operation instructions to manage the kbroker distributed operating system;
  • the management background When the management background receives the alarm or warning information reported by the kbroker distributed operating system, the alarm informs the operation and maintenance and developers for processing.
  • the kbroker distributed operating system extracts a kbroker_messager module for message forwarding:
  • the kbroker_messager module forwards messages based on the process number program_id;
  • Each kbroker_server module in the system communicates with a kbroker_messager module.
  • the kbroker_messager module saves the process number program_id of the kbroker_server module connected to it and the process number program_id of other program processes running on its server, and marks these process numbers program_id as belonging. Manage by yourself, and bind these process numbers program_id and its communication connection with the kbroker_server module;
  • the kbroker_messager module saves the process number program_id managed on other kbroker_messager modules connected to it, and binds the process number program_id and the corresponding kbroker_messager module with the communication connection between them;
  • the kbroker_super module manages the kbroker_messager module, and allocates each kbroker_server module to connect to a kbroker_messager module;
  • the kbroker_messager module When the kbroker_messager module is used for message forwarding, there is no communication connection between the kbroker_server modules, and the kbroker_server module forwards the message to the kbroker_messager module after receiving the message sent by the program process it manages.
  • the present invention also provides a storage medium on which a computer program is stored, and when the program is executed by a processor, a kbroker distributed operating system as described above is realized.
  • the present invention also provides an electronic device including a memory, a processor, and a computer program stored in the memory and capable of running on the processor.
  • the processor executes the program to implement the above-mentioned one kbroker distributed operating system.
  • Figure 1 shows a schematic diagram of the system structure of a kbroker distributed operating system, storage medium and electronic equipment in an embodiment of the present invention
  • Figure 2 shows a system startup flowchart of a kbroker distributed operating system, storage medium and electronic equipment in an embodiment of the present invention
  • FIG. 3 shows a flow chart of application startup of a kbroker distributed operating system, storage medium, and electronic device in an embodiment of the present invention
  • FIG. 4 shows a flowchart of starting the app_service module in the process of starting an application program of a kbroker distributed operating system, a storage medium, and an electronic device in an embodiment of the present invention
  • FIG. 5 shows a schematic diagram of the system structure of a kbroker distributed operating system, storage medium, and electronic equipment in an embodiment of the present invention.
  • the present invention provides a kbroker distributed operating system.
  • the kbroker distributed operating system is a distributed operating system applied to multiple servers, including: kbroker_server module, business layer module, and kbroker_super module , Gateway module.
  • kbroker_server module running on each server, which is the agent of the server, manages all functions on the server, including monitoring and reporting available resources, and is responsible for starting the app_allocator module and app_service module on the server and implementing it After startup, it connects with the kbroker_server module on the server through inter-process communication, closes and monitors the app_allocator module and app_service module running on the server, and creates and closes storage resources.
  • the kbroker_super module manages all servers through the kbroker_server module. Through the interconnection between kbroker_server modules, the resources on each server are concentrated into a large resource pool, and the kbroker_server module performs message forwarding according to the process number program_id of the target program process to realize message transmission between all programs in the system.
  • the kbroker_super module realizes the effective use of resources by selecting the appropriate kbroker_server module in the entire resource pool to start new programs and create storage resources.
  • the business layer logic is divided into several applications for implementation, and each application is composed of two corresponding modules, app_allocator and app_service.
  • the app_allocator module is used to manage the app_object objects of the application, including the creation, deletion, and allocation of app_object objects to specific app_service modules to run;
  • the app_service module is the object pool of app_object objects, responsible for the business logic allocated to the app_object objects running on it Specific implementation.
  • the app_allocator module applies to the kbroker_super module to start a new app_service module.
  • the business logic of the entire system is implemented by the logical processing of the app_object object running on the app_service module and the mutual calls between the app_object objects.
  • the management of the entire distributed operating system is divided into two layers: through the kbroker_super module and the kbroker_server module, the resources of all servers are integrated into a resource pool for use by the business-level applications; the app_allocator module and the app_service module are used to implement the application business Layer app_object object management.
  • the former realizes that when the server resources are insufficient, the resource pool can be directly expanded by adding a server to meet the demand.
  • the latter realizes that the business layer only needs to pay attention to the business itself and not the load problem (the number of app_object objects).
  • coroutine Making full use of the coroutine is a major feature of this system.
  • the purpose of this is two points: one is to make it not block the thread so that the logical processing of the app_object object can be queued and executed on a thread, which simplifies the complexity of logic development ;
  • the second is to realize the non-blocking call to the remote app_object object, so that the business layer calls the interface of the remote app_object object like calling the local interface, so that the entire system is like a single-threaded program to the developer.
  • the gateway module is a dedicated business layer module for receiving and responding to external requests.
  • the applications in the kbroker distributed operating system include: computing applications, whose business logic only involves computing and temporarily storing data in memory; storage applications, whose business logic involves using storage resources to save data.
  • Storage-type resources are storage media used to save data such as disks or third-party external programs to prevent data loss after the distributed operating system restarts.
  • Storage application refers to the app_service module, which is the application that uses storage resources in the business layer; the kbroker_super module and some app_allocator modules also use storage resources, which are called modules involving storage resources.
  • kbroker_super module selects the kbroker_server module for modules involving storage resources, it selects the kbroker_server module where the storage resources it uses instead of the kbroker_server module. The best kbroker_server module on the resource.
  • the system provides unified framework support for master-slave disaster recovery and message transfer support between master-slave, so that the business layer can complete master-slave synchronization by rewriting the data export and import interface and the data synchronization sending and processing interface.
  • All used storage resources are set with the resource number resource_id and saved by the kbroker_super module.
  • the association relationship between the storage resource module and the resource number resource_id of the storage resource needs to be saved by the appropriate module, and the master-slave between the modules
  • the disaster recovery relationship is realized by storing the combination relationship of the resource ID resource_id of the storage resource involved.
  • the kbroker_server module manages the specific generation of storage resources, including a local file used to save the connection and access methods of storage resources after generation.
  • the kbroker_server module passes it in when the program process is started.
  • the connection access method of storage resources allows the program process to connect and access the storage resources it needs; the kbroker_server module is also used to manage the destruction and deletion of storage resources.
  • the kbroker_super module includes a storage resource for storing server information, the correspondence between resource ID resource_id and the kbroker_server module, application information, and storage resources used by the kbroker_super module.
  • the server information includes the server's login information, the server number, and the command to start the kbroker_server module.
  • the application information includes the application number, application type, application parameters, and storage resources used by the app_allocator module of the application.
  • the kbroker_super module maintains and records the availability of CPU, memory, disk, and network on the server where the kbroker_server module is located, and selects the appropriate kbroker_server module for the newly allocated storage resources and the newly started program process based on this.
  • the program processes in the kbroker distributed operating system communicate through the process number program_id; the kbroker_server module stores the relationship between the process number program_id and the kbroker_server module on the server where it is located, and the message transmission in the system is forwarded by the kbroker_server module.
  • the kbroker_super module manages the server where it is located through the kbroker_server module.
  • the kbroker_super module realizes the addition and deletion of the server where the kbroker_server module is located in the kbroker distributed operating system by starting and closing the kbroker_server module on the server.
  • Each server used in the kbroker distributed operating system includes a unique kbroker_server module.
  • the kbroker_server module monitors the operating status of the server where it is located and the availability of CPU, memory, disk, and network, and reports to the kbroker_super module.
  • Multiple servers are interconnected through the network connection of the kbroker_server module running on it.
  • the kbroker_server module monitors the connection status and reports it to the kbroker_super module, and the kbroker_super module obtains and handles the abnormal operation of the kbroker_server module accordingly.
  • the kbroker_super module starts the kbroker_server module: After the kbroker_server module is started, it will read the network port it listens to from the configuration for monitoring. During the process of registering with the kbroker_super module, it will also upload the port it listens to as a parameter. The kbroker_super module After successful verification, the network address and listening port of kbroker_server will be saved.
  • the kbroker_server module After the kbroker_server module is successfully registered, it requests the kbroker_super module for the network address and listening port of other running kbroker_server, and then connects to other kbroker_server modules based on this information, and completes the connection after verification.
  • the kbroker_server module After the kbroker_server module is connected to another running kbroker_server module or is connected by other running kbroker_server modules, it will request the process ID program_id of all the program processes running on it, and after obtaining the program ID of these processes and its kbroker_server module Associate the program_id of the process number, and associate the forwarding rules of these process numbers program_id to the network connection between itself and its kbroker_server module.
  • the kbroker_server module After the kbroker_server module has started to complete the registration verification, and has connected with all other running kbroker_server modules and obtained the programs running on it, it notifies the kbroker_super module that the kbroker_server module is initialized successfully, and the kbroker_super module receives the notification of the successful initialization. Then, request the status of the resources available on it and put it in the available queue.
  • the kbroker_super module realizes adding and deleting the server where the kbroker_server module is located in the kbroker distributed operating system by starting and closing the kbroker_server module on the server, including: the kbroker_super module saves the login information of all servers, the number of the server, and the activation of the kbroker_server module Execute the command; the kbroker_super module uses the server login information to log in to the server; the kbroker_super module executes the startup command on the logged-in server to start the kbroker_server module; the started kbroker_server module connects to the kbroker_super module according to the information of the kbroker_super module in the startup command, and makes a request to the kbroker_super module Registration verification; the kbroker_super module verifies the registration request. If the verification is successful, the kbroker_server module starts successfully; the kbroker_super module deletes its server from the distributed operating system by notifying the
  • the kbroker_super module is also used to add a new server to the distributed operating system during the running process: submit the login information of the new server to the kbroker_super module and start the corresponding kbroker_server module command on the new server.
  • the kbroker_super module numbers the new server; the kbroker_super module Start and verify the kbroker_server module corresponding to the new server. If the verification is successful, the new server is added successfully, and the kbroker_super module saves the information of the new server.
  • the kbroker_super module selects the appropriate kbroker_server module for each app_allocator module of the application according to the number of app_allocator modules required by the application and starts the app_allocator module through the kbroker_server module; the kbroker_super module starts the application from the started application Select the main app_allocator module corresponding to the application in the app_allocator module, set the other started app_allocator modules of the application to the slave app_allocator module and complete synchronization with the main app_allocator module to achieve master-slave disaster recovery, and in the process of running Maintain the master-slave disaster recovery; the master app_allocator module starts the app_service module, and adds or reduces the number of app_service modules during operation according to the needs of the running resources.
  • the specific process of the app_allocator module to start the app_service module is as follows: the app_allocator module sends a request command to start an app_service module to the kbroker_super module; the kbroker_super module generates the process number program_id for the newly started app_service module, selects the appropriate kbroker_server module according to the start request, and sends it to the kbroker_server
  • the module sends the startup instruction to start the selected app_service module; the selected kbroker_server module starts the corresponding app_service module according to the startup instruction; after the app_service module is started, it connects to the kbroker_server module through system inter-process communication, and then completes the registration to the kbroker_server module according to the parameters in the startup process Verification.
  • the kbroker_server module After verifying the app_service module, the kbroker_server module replies to the kbroker_super module that the startup is successful, but does not add the app_service module to its own program process at this time; the kbroker_super module saves the newly started after receiving the successful reply from the kbroker_server module to start the app_service module
  • the process number program_id of the app_service module and its corresponding relationship with the kbroker_server module reply to the kborker_server module to confirm that the app_service module is successfully started and carry the request parameters of the app_allocator module; the kbroker_server module receives the message confirming that the app_service module is started successfully, and then the app_service module Add it to your own program process, and notify other kbroker_server modules that you have added this program process.
  • the kbroker_super module When the kbroker_super module starts the app_allocator module that uses storage resources: the kbroker_super module stores the relationship between the resource number resource_id of the storage resource used by the app_allocator module and the application number app_id of the application to which the app_allocator module belongs; the kbroker_super module starts its app_allocator module When using a storage resource application, find the resource number resource_id of all storage resources associated with the application number app_id of the application, find the corresponding kbroker_server module according to each resource number resource_id, and start the app_allocator that uses the storage resource Module, if there is no app_allocator module to start successfully, the application fails to start; if there is app_allocator module to start successfully, then determine the number of successful startups, if the number meets the needs of master-slave disaster recovery, complete and maintain the master-slave disaster recovery, otherwise start a new one The app_allocator module meets the number
  • the kbroker_server module is responsible for message forwarding in the system: all messages are forwarded through the message forwarding mechanism on the kbroker_server module.
  • the message forwarding mechanism is based on the process number program_id of the target program process; when the kbroker_server module receives a message to be forwarded, It will first find out the process number program_id of the message target program process, and then forward it according to the process number program_id.
  • the specific situations are as follows: when the process number program_id is owned by the current kbroker_server module, it is forwarded through the communication between the target program process and its own process; when the process number program_id is owned by other kbroker_server modules, it corresponds to the process number program_id
  • the kbroker_server module is forwarded to its own network connection; when the process number program_id does not exist, the message is directly discarded If the process number program_id has not been confirmed with the kbroker_super module within a short period of time, ask the kbroker_super module to confirm whether the process number program_id exists; if the process number program_id does not exist in the kbroker_super module, then the message If the sender owns it, the process number program_id is notified that it does not exist. If the sender is owned by other kbroker_server modules, the corresponding kbroker_server module is notified that it does not own the process number program_id.
  • the app_service module is managed by the app_allocator module corresponding to its application.
  • the kbroker_super module selects the appropriate kbroker_server module and the selected kbroker_server module completes the specific startup work.
  • the app_allocator module deletes or modifies the routing information cache of the app_object object running on the app_service module, and decides whether to start a new app_service module as needed.
  • app_allocator module will be idle app_service module.
  • the kbroker_server module is responsible for managing the app_allocator module and app_service module running on the server where it is located: the app_allcator module and app_service module are started by the kbroker_server module. After startup, they run on the same server as the kbroker_server module and connect to the kbroker_server module through system interprocess communication.
  • kbroker_server module monitors the abnormal status of app_allocator module and app_service module, reports kbroker_super module and handles it; kbroker_server module according to its server For the overall usage of CPU, memory, disk, and network, the app_allocator module and app_service module running on it are managed by overload setting, migration, and shutdown.
  • the kbroker_server module performs management operations such as overload setting, migration, and shutdown of the app_allocator module and app_service module running on it according to the overall resource usage of its own server.
  • management operations such as overload setting, migration, and shutdown of the app_allocator module and app_service module running on it according to the overall resource usage of its own server.
  • the kbroker_server module monitors that the resources used by its own server reach the set available threshold, it sets all app_service modules running on it to an overload state, so that it notifies its corresponding app_allocator module that no new app_object objects will be allocated to it.
  • the kbroker_server module monitors that the resources used by its own server reach the set limit threshold, it starts the migration work.
  • the computing app_service module is then notified to migrate the app_object object. If the app_object object supports migration (implements the export and import interface), it will be migrated to other idle app_service modules.
  • All storage-type resources will have a master-slave disaster recovery mode.
  • the slave app_allocator module and the slave app_service module in the master-slave disaster recovery mode running on it will be directly shut down.
  • the main function of the app_allocator module is to manage the app_object object, start enough app_service modules according to the running app_object object, and allocate these app_object objects that need to run to the appropriate app_service module to run, and at the same time
  • the system provides an interface for querying the routing information of the app_object object (which app_service module is running on).
  • the app_service module caches routing information, when the app_allocator module replies to a routing information request that the app_object object does not exist, it needs to record that the object does not exist once returned to the requester, so as to notify the requester to update when the app_object object is created. It is the route cache for the app_object object.
  • the app_object object of a computing application is divided into two modes: permanent existence and temporary existence: permanent existence: this kind of object always exists when it is created, unless it is deleted again; it does not exist when it is not created.
  • permanent existence this kind of object always exists when it is created, unless it is deleted again; it does not exist when it is not created.
  • Temporary existence type This is used for situations where the outside world does not care about or confirm specific objects, and the app_allocator module manages the creation and destruction of it as needed. For example, there are no restrictions on multi-person group shopping. Users only care about whether the group is successful and not the specific group.
  • the app_allocator module of the application the app_allocator module manages each group and will apply for it. The users participating in the group joining are assigned to the appropriate group joining.
  • the app_allocator module When an app_service module crashes, the app_allocator module will delete the routing information stored on all app_object objects running on the app_service module, and then allocate the corresponding app_object object when it encounters a routing information request for these app_object objects. Run on the new app_service module. In addition, it will consider whether to start a new app_service module as needed.
  • the app_allocator module is also used to create the app_object object of the management application: the app_object object of the computing application includes a permanent existence mode and a temporary existence mode, and the app_object object of the storage application is a permanent existence mode; the app_allocator module uses the app_service module Execute the creation and start the app_object object; the app_allocator module saves the routing information of the app_object object, the routing information is which app_service module the app_object object is running on, and provides an interface for requesting routing information; the app_allocator module provides the corresponding interface for the app_object object in the permanent mode
  • the function creates and deletes it; the app_object object in the permanent existence mode starts and runs when it is accessed, and the app_object object will be closed and exited if there is no access request for a long time; for the app_allocator module that manages the app_object object in the permanent existence mode, including A storage resource is used to save the existing app_object object; for the app_object object
  • the app_allocator module of the storage application maintains the master-slave disaster recovery of the app_service module.
  • the app_allocator module of the storage application compiles several corresponding app_service modules into a group, and sets the group number virtual_id for each group to maintain each group The master-slave disaster recovery of the app_service module.
  • the app_service modules of storage applications have corresponding storage resources.
  • the app_allocator module of storage applications saves the association between the group number virtual_id of the group and the resource number resource_id of the storage resource used by the app_service module in the group. .
  • the existing app_object object is bound to the group one-to-one, and the app_allocator module of the storage application saves the correspondence; a new app_object object selects a group to bind when applying for creation, and creates a new one when there is no available group Group and the app_service module in the group, and then select a group to bind.
  • the app_allocator module of the storage application processes the routing information request for the app_object object, after finding the group number virtual_id of the group corresponding to the app_object object, it finds the process number program_id of the main app_service module in the group corresponding to the group number virtual_id and returns to the request square.
  • the specific logic of the app_object object is uniquely identified by the application number app_id of the application and the object number object_id of the app_object object on the application.
  • the business layer uses the application number app_id of the application and the object of the app_object object on the application.
  • the numbered object_id accesses the app_object object;
  • the business layer implements business logic by rewriting the app_object_i virtual base class provided by the app_service module; the logic processing of a single app_object object can only run on a single app_service module, and a single app_service module can run multiple app_objects at the same time Object.
  • the kbroker distributed operating system provides a virtual interface that creates an app_object object through the object number object_id.
  • the app_service module creates an app_object object through the virtual interface rewritten by the business layer, and initializes the app_object object through the initialization interface of the app_object object.
  • the business logic of the same app_object object is queued for processing on the same thread on the app_service module.
  • Each task event of the app_object object is encapsulated into a coroutine for execution.
  • the kbroker distributed operating system provides coroutine support for blocking interfaces ,
  • the supported method is to replace the blocking interface of the operating system with a non-blocking interface that supports the coroutine after rewriting through the hook mechanism.
  • the app_service module caches the routing information of the app_object object, obtains the routing information through the app_allocator module corresponding to the app_object object to update the cache, and updates the cache of the source app_object object routing information in the access request when other app_object objects are accessed.
  • the external access of the business layer is based on the app_object object, and the system provides an interface for accessing the external app_object object, which is implemented by the app_service module; the app_service module finds the process number program_id of the app_service module where the app_object object is running according to the cache or routing information requested from the corresponding app_allocator module, Sending a message whose destination is the process number program_id causes the message to be sent to the app_service module where the app_object object is running.
  • the app_service module provides coroutine support for remote function calls that access external app_object objects.
  • the app_service module After the app_service module receives the pending message of the app_object object, it checks the operating status of the target app_object object and processes it. The app_service module notifies the app_allocator module to no longer allocate new app_object objects to run on it when it monitors its overload.
  • the app_service module provides an interface for the business layer to add asynchronous events and delayed events, coroutine locks and coroutine signals.
  • the app_service module supports migrating the running app_object object to other app_service modules in the same application.
  • the app_service module of the storage application provides a master-slave disaster recovery framework for the app_object object, and the master-slave synchronization-related interface in the app_object_i virtual base class rewritten by the business layer realizes the master-slave disaster recovery function.
  • the app_service module of the storage application Provides an interface for transferring data between master and slave app_object objects.
  • the app_service module is a module that implements specific business logic.
  • the other modules of the system are to provide support for it, so that the entire system can run a large number of app_service modules of various applications, and then support a large number of app_object objects to run at the same time, and the entire distributed system Compared to developers, it becomes a simple single-threaded program.
  • the app_service module is an object pool of app_object objects.
  • the object is created through the object creation interface rewritten by the business layer, and the message of a certain app_object object sent to the module is passed to the corresponding app_object object running on it for execution.
  • the app_service module When the app_service module receives a business layer message, it will first determine the status of the target app_object object: the target app_object object is in a normal state, and directly call the interface of the app_object object to execute; the target app_object object is being initialized, then the message is cached, and the initialization is completed and then executed in sequence ;
  • the target app_object object is migrating, the message is cached, and later if the migration is successful, it will be delivered to the app_service module after the migration is executed; if the migration fails, the app_obect object will be restored to the normal state, and the cached messages will be executed in sequence; the target app_object If the object is not on itself, it will look up the routing information of the target app_object object and forward the message.
  • the system provides the notify_api_call interface and function_api_call interface for accessing the remote object for the business layer, both of which use the application number app_id and the object number object_id of the target app_object object as parameters to access the remote object.
  • the notify_api_call interface does not need to return a value, only the message can be delivered; the function_api_call interface is a coroutine remote function call. After the message is delivered, the stack is saved to give up the CPU, and then the remote app_object object returns the message before restoring the stack to get the return. Value continues to execute.
  • the entire distributed system is for developers to develop large-scale distributed server-side programs like the simplest single-threaded programs. Therefore, many places have done the processing to adapt to conventional thinking.
  • the function_api interface of the app_object object corresponds to the total entry of the function that needs to return a value in the class;
  • the notify_api interface of the app_object object corresponds to the total entry of the function that does not require a return value in the class;
  • the notify_api_call interface will remotely call the object that does not require a return value.
  • the function interface is converted into message delivery.
  • the function_api_call interface coroutines the function interface of the remote call object that needs to return the value.
  • the business layer can use the remote call just like the local call.
  • the business layer application uses the notify_api_call interface and the function_api_call interface to encapsulate the interface calls provided by the object into a header file, and other business layers directly use these encapsulated function interfaces to call the function interface of the object.
  • the encapsulated header file is similar to the class header file in traditional programs. In order to allow developers to use existing design patterns, we provide a singleton for the current app_object object-object_singleton. Developers can inherit object_singleton to use it like the traditional singleton pattern.
  • the app_service module When the app_service module implements the function of adding asynchronous events and delayed event interfaces, it cannot be added directly when the business layer calls the interface. Instead, these events should be cached and added after the execution of the current task logic is completed, so as to avoid the current logic has not been executed. The problem of the execution of the added asynchronous event or delayed event.
  • the app_service module also provides an interface for directly adding asynchronous events in order to cooperate with the coroutine signal.
  • the coroutine lock provided by the app_service module will cache the stack when other tasks have been locked, give up the CPU, and resume the stack after other tasks release the lock.
  • the return value can be used to determine whether it was interrupted and then resumed. Executed, and judge whether it is a timeout exit through the wrong parameter.
  • the coroutine signal provided by the app_service module mainly has two functions: monitoring and sending. During monitoring, the stack is cached to give up the CPU, and the stack is resumed to continue execution after other tasks send corresponding signals.
  • the business layer implements business logic by rewriting the app_object_i virtual base class.
  • the app_object_i virtual base class of computing applications and the app_object_i virtual base class of storage applications include function_api interface, notify_api interface, initialization interface, delete interface, and export data interface. , Import data interface; function_api interface is used to process access requests that require a return value, and notify_api interface is used to process access requests that do not require a return value.
  • the business layer module implements business logic processing by rewriting the function_api interface and notify_api interface; the business layer module rewrites the initialization interface for the app_service module to perform initialization after creating the app_object object; the business layer module rewrites the delete interface for the app_service module when the app_object object is closed Release the data created by the business layer; the export data interface and the import data interface are used for the migration of app_object objects. By default, app_object object migration is not supported. If the business layer rewrites these two interfaces, the application supports the migration of app_object objects. When the business load is high, the app_object object is migrated to the app_service module with low load.
  • the app_service module of storage applications implements master-slave disaster recovery, and its app_service module implements the overall export and import interface and master-slave switching interface for the data of its storage-type resources.
  • its app_object_i virtual base class also includes a master-slave synchronization interface, master-slave The synchronization interface cooperates with the master-slave disaster recovery framework of the app_service module to realize the master-slave disaster recovery of the logical data of the business layer.
  • the gateway module includes a session_allocator module and a kbroker_gateway module; the session_allocator module is developed based on the basic functions of the app_allocator module, saves the socket monitoring information of the kbroker_gateway module, and realizes load balancing of multiple kbroker_gateway modules.
  • the kbroker_gateway module is extended and developed based on the basic functions of the app_service module, and supports various network-based external request access methods.
  • the app_object object running on it is a passively created external request connection that includes the object number object_id.
  • the kbroker_gateway module is used to receive external requests and parse the request from the request to obtain the application number app_id and object number object_id corresponding to the target app_object object.
  • the kbroker_gateway module supports setting default message processing strategies for external requests and special processing strategies based on the application number app_id of the target app_object object corresponding to the request.
  • the processing strategy is determined by the application number app_id of the target app_object object parsed from the external request. If there is a special processing strategy, the special processing strategy is executed, otherwise the default processing strategy is executed.
  • the kbroker_gateway module also includes the notify_api interface and function_api interface that provide access to other applications in the system; the notify_api interface and function_api interface constitute the message processing strategy in the system, and the message processing strategy in the system is the default message processing strategy or an application based on the message source app_object object.
  • the special message processing strategy of the program number app_id supports the setting of a special message processing strategy for a certain application number app_id.
  • the gateway module mainly receives external requests, parses out the target app_object object from the request, and then sends the request to the target app_object object for processing.
  • the specific operation also supports special processing logic for the request. From the type of gateway, it can be divided into long connection type and short connection type.
  • Persistent connection requests usually require login verification.
  • the session_allocator module is responsible for processing login verification. After verification is successful, there will be an internal object_id, so you can directly bind the connection to the object_id, and the subsequent operations will be the same as the regular app_object It's similar.
  • the most typical short connection type request is the http request.
  • the http request is similar to a functional call, one request is returned.
  • the function_api_call interface is used to access the target app_object object. After the function returns, the return data is sent to the socket connection.
  • the gateways of the entire distributed system can be divided into service layer gateways, system management gateways, and service management gateways in terms of function types.
  • the service layer gateway is a gateway provided by the service layer to receive external requests and is used to implement the external access interface provided by the system service layer.
  • the system management gateway is a gateway interface provided to system administrators. System administrators can obtain information such as current system operation and fault alarms through these gateway interfaces, and can also perform some operations through these interfaces (including but not limited to adding or deleting servers, compulsory Close an application, etc.) to maintain the operation of the system.
  • the business management gateway is a gateway submitted to developers for use. Its functions include querying the running status and alarm information of applications, adding and deleting applications, and publishing applications.
  • the kbroker distributed operating system includes a singleton application.
  • the singleton application manages the singleton type business logic in the kbroker distributed operating system.
  • the singleton type business logic cannot be distributed to multiple places but only in one program process.
  • the business logic executed in.
  • the singleton application is a special application in the kbroker distributed system.
  • the application number of the singleton application is the singleton_app_id assigned by the system.
  • the app_service module of the singleton application has various functions.
  • the program process of singleton business logic is a special application in the kbroker distributed system.
  • the app_allocator module of the singleton application assigns a singleton_object_id to each app_service module, which is used to display the app_service module as an object of the singleton application relative to other applications.
  • the business layer uses the singleton application numbers singleton_app_id and app_service
  • the singleton object number singleton_object_id of the module accesses the app_service module of the singleton application.
  • the app_service module of a singleton application supports master-slave disaster recovery, and the app_allocator module manages and maintains the master-slave disaster recovery of the app_service module.
  • the app_service module of a singleton application includes a unique business layer app_object object, and all business layer requests are handled by this app_object object.
  • the app_allocator module of the singleton application is provided by the kbroker distributed operating system, and the kbroker_super module and the kbroker_server module provide support for starting the singleton application.
  • the singleton application is to solve the logic that can only be globally unique, such as the common id generator.
  • the id generator provides the ID number generation work for the entire system. There can only be one such program to run the logic in the entire system.
  • the realization of functional requirements is very different from the previous ordinary applications that are allocated to multiple app_service modules to execute logic operations. For this reason, a special type of application needs to be provided to support this functional requirement.
  • the kbroker_super module supports master-slave disaster recovery, and the kbroker_super module implements a master-slave disaster recovery mechanism with one master and multiple slaves.
  • the master kbroker_super module starts a new slave kbroker_super module to implement replacement;
  • all slave kbroker_super modules are elected to determine one of the slave kbroker_super modules to switch to the master kbroker_super module, and then start the new slave kbroker_super module Achieve replacement.
  • the kbroker distributed system starts the master kbroker_super module, and the master kbroker_super module starts the slave kbroker_super module, and completes data synchronization to achieve master-slave disaster recovery.
  • the main kbroker_super module sets different priorities for all kbroker_super modules. When the main kbroker_super module fails, the remaining available slave kbroker_super modules select the currently available highest priority kbroker_super module as the main kbroker_super module.
  • the master-slave disaster recovery of the above kbroker_super module does not depend on other existing programs or libraries for independent implementation, and can also be implemented with the aid of open source software such as zookeeper.
  • the kbroker distributed operating system also includes a management background for operating the kbroker distributed operating system.
  • the management background provides an access interface to the kbroker distributed operating system.
  • the kbroker distributed operating system obtains the access interface at startup through configuration or startup parameters; the kbroker distributed operating system reports status information through the interface, and the status information includes the connection information of the management gateway , The monitoring information of server operation, the running status of each application, the alarm for abnormal conditions, and the early warning for specific conditions.
  • the management background obtains the operating information of the kbroker distributed operating system through the management gateway and sends operation instructions to manage the kbroker distributed operating system.
  • the management background When the management background receives the alarm or warning information reported by the kbroker distributed operating system, the alarm informs the operation and maintenance and developers for processing.
  • the kbroker distributed operating system When supporting larger-scale server clusters, the kbroker distributed operating system extracts a kbroker_messager module for message forwarding, and the kbroker_messager module performs message forwarding based on the process number program_id.
  • Each kbroker_server module in the system communicates with a kbroker_messager module.
  • the kbroker_messager module saves the process number program_id of the kbroker_server module connected to it and the process number program_id of other program processes running on its server, and marks these process numbers program_id as belonging. Manage by yourself, and bind these process numbers program_id and its communication connection with the kbroker_server module.
  • the communication connection between multiple kbroker_messager modules, the kbroker_messager module saves the process number program_id managed on other kbroker_messager modules connected to it, and binds the process number program_id with the communication connection between the kbroker_messager module and the kbroker_messager module.
  • the kbroker_super module manages the kbroker_messager module, assigning each kbroker_server module to a kbroker_messager module.
  • the kbroker_messager module When the kbroker_messager module is used for message forwarding, there is no communication connection between the kbroker_server modules, and the kbroker_server module forwards the message to the kbroker_messager module after receiving the message sent by the program process it manages.
  • the kbroker_super module and the kbroker_server module are merged into a kbroker program.
  • the kbroker_super module is available on the server where the program is located, the kbroker_super module in the program is started, and if there is no kbroker_super module, it is not started. In this way, only one key kbroker program is needed on the entire distributed system.
  • the distributed system provides library files and interface files for these two modules, app_allocator module and app_service module. The application program rewrites these interfaces and then compiles and links with the library files to form the real app_allocator of the application. Procedures and app_service procedures.
  • the kbroker_super module has a storage resource to store the login information of all servers, the kbroker_server number and the command to start the kbroker_server module; in addition, it stores the application number app_id of the used application; it also stores the resource number resource_id of all resources and their location The association relationship of the kbroker_server module on the server.
  • the kbroker_super module is a part of the kbroker program and runs on a set of independent threads of the kbroker program.
  • the main functions of the kbroker_super module are as follows:
  • the process number program_id For each running program, including the kbroker program where the kbroker_server module is located, the app_allocator program and app_service program of the application, and the kbroker_super module itself, the process number program_id is allocated. At the same time, the process number program_id of the kbroker program process is used as the server number, and the correspondence between all process numbers program_id and the server number where it is located is saved.
  • the kbroker_server module in the kbroker program is mainly to connect to other servers in the system and manage the programs on your own server.
  • the core functions are as follows:
  • Listening to a network port is used to accept connections from kbroker programs on other servers.
  • the kbroker program When the kbroker program is connected to other kbroker programs, it will send its own number and check code to the other party. The other party will take this information and the network address of the source kbroker program to the kbroker_super module for verification. After the verification is successful, the two parties are connected successfully, and after the connection is successful Both parties will obtain each other's managed process ID program_id and bind the message delivery of these process ID program_id to the network connection of both parties.
  • the kbroker_server module will save all the process numbers program_id, and the corresponding relationship between these process numbers program_id and the process number program_id of the kbroker program process to which they belong, and the unknown process number program_id or the process number program_id and the process number of the kbroker program process are encountered during the running process.
  • the kbroker_super module will decide and deal with conflicts in the correspondence between program_id.
  • the kbroker_server module will start the corresponding program after receiving the command to start the program of the kbroker_super module.
  • it When starting the app_allocator module or app_service module of an application, it will first check whether the execution file of the corresponding application exists, and it will directly Start the executable file; if it does not exist, download the software package corresponding to the application from the network.
  • the software package contains all the executable files and configuration of the application. After downloading, decompress, and then start the executable file.
  • the program is started, it is connected to the kbroker program through the local socket of the operating system.
  • the kbroker_server module of the kbroker program saves the process number program_id of the newly launched program process and the corresponding relationship between its socket connection after the kbroker_super module confirms that the program is successfully started, and the corresponding relationship between its own socket
  • the new process number program_id is broadcast to the kbroker_server module of other kbroker programs.
  • the kbroker program has a thread dedicated to message forwarding. All messages are forwarded according to the target process number program_id. If the process number program_id is on other servers, it is forwarded to the kbroker_server module of the corresponding server through the socket. If the process number program_id is on its own server , Then directly find the corresponding socket and send it to the corresponding program.
  • the kbroker program is responsible for monitoring the resource status of the server where it is located and the status of the program process on it, and manages and maintains the socket connection status with other kbroker programs.
  • the kbroker program uses scripting languages such as python to implement some relatively cumbersome non-core functions, such as the startup of app_allocator and app_service programs, and server resource monitoring.
  • the app_allocator module provides an app_allocator class library for the business layer to use, provides some interfaces for the business layer to implement its own proprietary logic, and then combines the proprietary code and app_allocator class library to compile the application-specific app_allocator program.
  • the core function of the app_allocator module class library is to manage the allocation of app_object objects, and manage the app_service module that runs the app_object objects. These functions have been introduced in the previous section. Pay attention to the same request-such as routing request-in the specific implementation. In the case of multiple transmissions, in this case, a locking mechanism can be used to avoid repeated operations.
  • the app_allocator module class library provides processing interfaces for core operations such as incoming routing query requests, the successful startup and shutdown of the app_service module, and the successful startup or shutdown of the app_object object.
  • the business layer can implement its own proprietary logic by rewriting these interfaces, and You can specify whether to execute the regular logic provided by the class library after executing your own proprietary logic.
  • the app_allocator module does not have many types.
  • some proprietary programs can be appropriately packaged, such as the dedicated app_allocator program for common permanent objects, mysql, file storage, scene entry, etc. to simplify Development.
  • the app_service module provides an app_service class library for the business layer to use.
  • the business layer implements the specific functions of the business layer by rewriting the virtual base class of app_object_i, and rewrites the virtual interface provided by the system to create the app_object object to create real business layer objects. .
  • the business layer of the app_service module does not involve any content related to the server or the process number program_id, only the app_object object.
  • the app_object object is accessed through the application number app_id and the object number object_id.
  • the parameters of the notify_api and function_api interfaces have cmd command words and msg message content.
  • notify_api interface does not require a return value, it may send messages to the source app_object object during execution. Therefore, the parameters of the notify_api interface need to have the application number app_id and the object number object_id of the source app_object object of the message as parameters.
  • the function_api interface should perform the same logic for all request sources, so there is no need to distinguish which specific app_object object is the request source, but the parameter can have the application number app_id of the source app_object object of the message, and the application number app_id is for Do access control.
  • notify_api_call interface, function_api_call interface, coroutine locks and signals, adding asynchronous events and delayed events and other interfaces are unified into a user_api header file, and the business layer refers to the header file and uses related interfaces when needed.
  • the app_service module class library provides basic class libraries, and can also provide proprietary class libraries for common needs such as mysql to simplify development. All these class libraries provide their own setting interfaces for business layer settings. Write the interface and other parameters for creating business layer objects.
  • the gateway module is used to monitor network connections, receive external requests and forward them to the corresponding internal app_object object for processing, and send the returned content to the requester. It also provides a kbroker_gateway module class library for the business layer to use.
  • the kbroker_gateway module class library provides a network monitoring interface for the business layer to set the network monitoring port.
  • the kbroker_gateway module class library provides a virtual base class for network processing strategies.
  • the virtual base class includes an interface for handling strategies for events such as new connection arrival, message receipt, connection disconnection, etc.; and an interface for setting strategies.
  • the business layer rewrites this virtual base class to implement proprietary logic, and sets up the rewritten policy through the policy setting interface.
  • the kbroker_gateway module class library provides an interface for setting the default processing strategy for client messages, and an interface for special processing strategies based on the application number app_id of the message target app_object object.
  • the kbroker_gateway module class library provides an interface for setting a proprietary processing strategy for messages from other programs in the system.
  • the strategy is based on the application number app_id of the app_object object of the message source.
  • the specific interface in the processing strategy is the notify_api interface and the notify_api interface of the app_object object.
  • the function_api interface is the same. If there is no proprietary processing strategy, the default strategy is used. In the default strategy, the notification_api interface processing is sent to the client through the network connection of the target app_object object, and the function_api interface processing is to directly return an error.
  • An application program will eventually have its dedicated app_allocator program and app_service program, and its configuration file, which is packed into a compressed package, which includes the above files and a general configuration file.
  • This general configuration file will specify the app_allocator program And the start command of the app_service program, this compressed package will be uploaded to an internal server for the entire system to download.
  • the kbroker program on each server can download its corresponding program package through the application number app_id of the application when needed, and then After decompressing to the local, start the program in the application.
  • the kbroker_super module, app_allocator module and the app_service module of storage applications have master-slave disaster recovery. If the master-slave is not specified in the above description, it will default to the master module of the corresponding module. Only the master module will be responsible for logic processing, and the slave module will not Participating in logic processing is only responsible for synchronously saving the state of the main module.
  • the present invention also provides a storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned kbroker distributed operating system is realized.
  • the present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored on the memory and capable of running on the processor.
  • the processor program is in progress to realize the above-mentioned kbroker distributed operating system.
  • a kbroker distributed operating system, storage medium and electronic equipment of the present invention developers only need to focus on their own business and develop large-scale server-side programs like the simplest single-threaded program without paying attention.
  • a series of problems caused by the increase in traffic. Improve the efficiency of server clusters, and provide disaster tolerance mechanisms for software and hardware failures to achieve high availability. Therefore, the present invention effectively overcomes various shortcomings in the prior art and has a high industrial value.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Quality & Reliability (AREA)
  • Stored Programmes (AREA)

Abstract

本发明的一种kbroker分布式操作系统、存储介质和电子设备，应用于多台服务器，包括多个app_service模块和多个app_allocator模块，app_service模块用于运行业务逻辑，app_allocator模块用于管理应用程序的运行资源，并依据运行资源的需求量分配运行应用程序的app_service模块；包括多个kbroker_server模块，用于管理分布式操作系统中对应的服务器、以及开启和关闭对应服务器上的app_service模块和app_allocator模块，多个kbroker_server模块之间数据通信连接；包括kbroker_super模块用于管理整个系统内的所有程序进程。在本发明的支持下，开发者只用关注于自身业务本身，像开发最简单的单线程程序一样开发大型服务器端程序，不需要关注随访问量增加而导致的一系列问题，提高了服务器集群效率，系统自身提供了针对软硬件故障的容灾机制以实现高可用性。

Description

一种kbroker分布式操作系统、存储介质和电子设备 技术领域
本发明涉及服务器端的软件开发和运维系统领域，特别是涉及一种kbroker分布式操作系统、存储介质和电子设备。
背景技术
当今服务器端程序支撑的负载越来越大，大用户量高并发的需求越来越普遍，这就导致很多的互联网初创企业会遇到技术瓶颈问题，业务发展的越好越快对技术的挑战就越大，如何能提供一套简单可行的方案来帮助互联网初创企业解决技术瓶颈，从而让互联网初创企业的业务发展不受自身技术制约就成了一个很普遍而迫切的需求。
目前，对于大型服务器端程序的开发，主要技术解决方案有微服务，消息队列，定时程序，通常情况下需要三种方案组合使用才能得到比较好的效果。对于服务器运维方面比较先进的是基于kubernetes管理的docker技术。
微服务就是把整个大功能拆分成多个相对独立的小功能，微服务在程序设计时同样需要考虑随着访问量的增加而导致的大负载问题，从这个角度来说微服务仍然是一个应付大负载高并发的课题，只是从一个复杂的大功能缩减到一个相对简单的小功能而已，另外功能拆分简单的同时也增加了不同微服务之间调用的复杂度以及微服务整体系统运行治理的难度。
消息队列是为了应付瞬间高负载时的削峰、消息处理异步化和实现功能之间的解耦，消息队列虽然有开源程序可以使用，但业务逻辑代码编写需要把原本一个逻辑分拆成消息的生产者和消息的执行者两个逻辑，额外部署程序加代码拆分会导致大规模使用时，开发和运维都不是一件轻松的事情。
定时程序是为了解决在没有外部访问触发的情况下定期执行的任务，通常情况下程序需要用到的延时任务也会用定时任务的方式在对应的时间点触发。定时任务需要在服务器端配置，另外还需要专门的代码来保存待处理的任务以及编写任务触发时回调函数的具体处理，如果有很多种不同的定时任务，那对开发和运维来说都很麻烦而且容易出错。
基于kubernetes的docker技术可以简单的理解为一种简化的虚拟机方案，docker比虚拟机更轻量级一些，主要是解决程序部署运行方面的难题，尤其是程序运行环境的差异问题，但要把这套运维体系运行起来对运维人员的能力要求还是比较高的，而且还需要从代码层面做一定的适配开发才可以，要玩转这套体系挑战还是很大的。即使我们将其考虑成更简单一 些的云服务(例如阿里云)加自动化部署，但也需要比较强的运维能力才行
另外在运营维护方面，如何应对服务器硬件故障和软件故障来保证高可用性也是比较麻烦的，这需要从服务器运维和开发层面同时做出可行方案才可以。把这三个方案从开发和运维都实现和做好的难度是相对高的，再加上还需要处理服务器集群的软硬件故障问题，这也是为什么大多数小公司在面对大并发访问时服务器端程序会出问题的原因。
发明内容
鉴于以上所述现有技术的缺点，本发明的目的在于提供一种kbroker分布式操作系统和存储介质，用于解决现有技术中的问题：
第一，代码开发方面利用微服务做功能拆分，利用消息队列来做功能间解耦，再加上定时延时程序来完成开发，这样的开发模式用到的框架组件多，系统复杂，对开发者的要求高；
第二，运维方面利用基于kubernetes的docker技术应对负载增加和搭建高可用性的容灾架构对技术要求都比较高。
为解决上述技术问题，本发明是按如下方式实现的：一种kbroker分布式操作系统，包括：
kbroker_server模块，用于管理分布式操作系统中其所在的服务器，kbroker_server模块管理其所在服务器上的app_service模块和app_allocator模块、存储型资源；多个kbroker_server模块之间数据通信连接；
业务层模块，用于实现整个分布式操作系统的业务逻辑；所述业务逻辑被拆分成若干个应用程序；每个应用程序包括与之对应的一组app_allocator模块和至少一组app_service模块，app_service模块用于运行所述业务逻辑，app_allocator模块用于管理应用程序的运行资源，所述运行资源为该应用程序执行其业务逻辑所需要的服务器资源；app_allocator模块通过所述运行资源的需求量分配运行应用程序的app_service模块；所述业务逻辑由运行在app_service模块上的业务层app_object对象的逻辑处理以及app_object对象之间的相互调用实现；
kbroker_super模块，用于通过kbroker_server模块管理kbroker_server模块所在的服务器；通过kbroker_server模块管理分布式操作系统中的所有程序进程，并为每个程序进程设置进程编号program_id；通过kbroker_server模块管理分布式系统上所有已分配使用的和可用的存储型资源，并为每个已分配使用的存储型资源设置资源编号resource_id以及保存资源编号resource_id与所属kbroker_server模块的对应关系；管理业务层应用程 序，并为每个业务层应用程序设置应用程序编号app_id；
网关模块，为专用的业务层模块，用于接收和响应外部请求。
进一步地，分布式操作系统中的应用程序包括：
计算型应用程序，其业务逻辑只涉及到运算和临时在内存中保存数据；
存储型应用程序，其业务逻辑涉及到使用存储型资源保存数据。
进一步地，存储型资源为用于保存数据防止kbroker分布式操作系统重启后丢失数据的存储介质；
kbroker_server模块管理存储型资源的具体生成工作，包括一个本地文件用于生成后保存存储型资源的连接访问方式，当有程序进程用到该存储型资源时，kbroker_server模块在启动程序进程时传入该存储型资源的连接访问方式，使得程序进程连接并访问其需要的该存储型资源；kbroker_server模块还用于管理存储型资源的销毁删除工作。
进一步地，kbroker_super模块包括一存储型资源，用于保存服务器信息、资源编号resource_id与kbroker_server模块的对应关系、应用程序的信息、kbroker_super模块所用到的存储型资源。
进一步地，kbroker_super模块维护并记录所有kbroker_server模块所在服务器上的CPU、内存、磁盘、网络的可用状况，并据此为新分配的存储型资源和新启动的程序进程选择kbroker_server模块。
进一步地，kbroker分布式操作系统中的程序进程之间通过进程编号program_id进行通信；kbroker_server模块保存进程编号program_id和其所在服务器上的kbroker_server模块的关联关系，系统中的消息传递由kbroker_server模块转发。
进一步地，kbroker_super模块通过kbroker_server模块管理其所在的服务器：
kbroker_super模块通过启动和关闭服务器上的kbroker_server模块，实现向kbroker分布式操作系统内添加和删除该kbroker_server模块所在的服务器；
每台kbroker分布式操作系统内使用的服务器上包括一个唯一的kbroker_server模块；
kbroker_server模块监控其所在服务器的运行状况和CPU、内存、磁盘、网络的可用情况，并上报给kbroker_super模块；
多个服务器之间通过在其上运行的kbroker_server模块的网络连接来实现互联，kbroker_server模块监控连接的状况并上报给kbroker_super模块，kbroker_super模块据此得到kbroker_server模块的运行异常情况并做处理。
进一步地，kbroker_super模块通过启动和关闭服务器上的kbroker_server模块来实现 向kbroker分布式操作系统内添加和删除该kbroker_server模块所在的服务器，具体包括：
kbroker_super模块保存所有服务器的登录信息、服务器的编号、启动kbroker_server模块的执行命令；
kbroker_super模块利用服务器登录信息登录服务器；
kbroker_super模块在登录的服务器上执行启动命令启动kbroker_server模块；
启动后的kbroker_server模块根据启动命令中的kbroker_super模块的信息连接到kbroker_super模块，并向kbroker_super模块请求注册校验；
kbroker_super模块校验注册请求，校验成功则kbroker_server模块启动成功；
kbroker_super模块通过通知已校验成功的kbroker_server模块自行关闭实现从分布式操作系统中删除其所在服务器。
进一步地，kbroker_super模块还用于在运行过程中向分布式操作系统中添加新服务器：
向kbroker_super模块提交新服务器的登陆信息以及在新服务器上启动对应的kbroker_server模块命令；
kbroker_super模块对新服务器进行编号；
kbroker_super模块启动并校验新服务器对应的kbroker_server模块，如果效验成功，则添加新服务器成功，同时kbroker_super模块保存新服务器的信息。
进一步地，当应用程序启动时：
kbroker_super模块根据该应用程序需要的app_allocator模块数量，为该应用程序的每个app_allocator模块选择kbroker_server模块并通过kbroker_server模块启动该app_allocator模块；
kbroker_super模块从已启动的该应用程序的app_allocator模块中选择确定应用程序对应的主app_allocator模块，将该应用程序其他已启动的app_allocator模块设置成从app_allocator模块并与主app_allocator模块之间完成同步，实现主从灾备，并在运行过程中维护主从灾备；
主app_allocator模块启动app_service模块，并在运行过程中根据运行资源的需要新增或者缩减app_service模块的运行数量。
进一步地，kbroker_super模块在启动用到存储型资源的app_allocator模块时：
kbroker_super模块保存app_allocator模块使用的存储型资源的资源编号resource_id和app_allocator模块所属应用程序的应用程序编号app_id的关联关系；
kbroker_super模块在启动其app_allocator模块用到存储型资源的应用程序时，找到 该应用程序的应用程序编号app_id关联的所有存储型资源的资源编号resource_id，根据每个资源编号resource_id找到对应的kbroker_server模块并启动使用该存储型资源的app_allocator模块，如果没有app_allocator模块启动成功则应用程序启动失败；如果有app_allocator模块启动成功则再判断启动成功的数量，如果数量满足主从灾备的需要则完成并维护主从灾备，否则先启动新的app_allocator模块满足主从灾备的数量，然后再完成并维护主从灾备；
当kbroker_super模块启动一个新的app_allocator模块时，创建存储型资源，通过此存储型资源启动新的app_allocator模块，kbroker_super模块保存新创建的存储型资源的资源编号resource_id和app_allocator模块所属应用程序的应用程序编号app_id的关联关系；
当使用中的app_allocator模块不可用时，删除其使用的存储型资源以及该存储型资源的资源编号resource_id和app_allocator模块所属应用程序的应用程序编号app_id的关联关系，然后启动一个新的app_allocator模块替换掉不可用的app_allocator模块。
进一步地，app_service模块由其应用程序对应的app_allocator模块管理：
当app_allocator模块启动app_service模块时，通过kbroker_super模块选择kbroker_server模块并由被选择的kbroker_server模块来完成具体的启动工作；
当app_service模块不可用时，app_allocator模块删除或修改该app_service模块上运行的app_object对象在其上的路由信息缓存，根据需要决定是否启动新的app_service模块；
当app_service模块过多时，app_allocator模块将关闭空闲的app_service模块。
进一步地，kbroker_server模块负责管理其所在服务器上运行的app_allocator模块和app_service模块：
app_allcator模块和app_service模块由kbroker_server模块来执行启动操作，启动后和kbroker_server模块跑在同一台服务器上并通过系统进程间通信连接于kbroker_server模块；
app_allocator模块和app_service模块与kbroker分布式操作系统内其他程序进程的消息传递均由kbroker_server模块进行转发；
kbroker_server模块监控app_allocator模块和app_service模块的异常状况，上报kbroker_super模块并处理；
kbroker_server模块根据其所在服务器CPU、内存、磁盘、网络的总体使用情况，对其 上运行的app_allocator模块和app_service模块进行过载设置、迁移、关闭的管理操作。
进一步地，app_allocator模块还用于创建管理应用程序的app_object对象：
计算型应用程序的app_object对象的模式包括永久存在模式和临时存在模式，存储型应用程序的app_object对象的模式为永久存在模式；
app_allocator模块通过app_service模块执行创建启动app_object对象；
app_allocator模块保存app_object对象的路由信息，路由信息为app_object对象运行在哪个app_service模块上，并对外提供请求路由信息的接口；
app_allocator模块对于永久存在模式的app_object对象，提供相应的接口函数对其进行创建和删除；
永久存在模式的app_object对象当被访问时启动运行，长时间没有访问请求的情况下app_object对象将被关闭并退出运行；
对于管理永久存在模式app_object对象的app_allocator模块，包括一个存储型资源用于对已存在的app_object对象进行保存；
对于临时存在模式的app_object对象，由app_allocator模块根据需要自行分配；
app_allocator模块通过添加专属功能满足应用程序对app_object模块的特定要求。
进一步地，存储型应用程序的app_allocator模块维护app_service模块的主从灾备：
存储型应用程序的app_allocator模块把若干个相应的app_service模块编成一组，并对每个分组设置分组编号virtual_id，维护每个分组内app_service模块的主从灾备；
存储型应用程序的app_service模块都设有与之对应的存储型资源，存储型应用程序的app_allocator模块保存分组的分组编号virtual_id与该分组内app_service模块用到的存储型资源的资源编号resource_id的关联关系；
已存在的app_object对象与分组一一对应绑定，存储型应用程序的app_allocator模块保存该对应关系；新的app_object对象在申请创建时选择一个分组进行绑定，当没有可用的分组时先创建新的分组和分组内的app_service模块，然后再选择一个分组进行绑定；
存储型应用程序的app_allocator模块在处理对app_object对象的路由信息请求时，查找到该app_object对象对应分组的分组编号virtual_id后，找到该分组编号virtual_id对应分组内的主app_service模块的进程编号program_id返回给请求方。
进一步地，app_service模块运行app_object对象的具体逻辑：
app_object对象由应用程序的应用程序编号app_id以及app_object对象在应用程序上的对象编号object_id来唯一标识，业务层通过应用程序的应用程序编号app_id以及 app_object对象在应用程序上的对象编号object_id对app_object对象进行访问；
业务层通过重写app_service模块提供的app_object_i虚基类来实现业务逻辑；
单个app_object对象的逻辑处理只能运行在单个app_service模块上，单个app_service模块上同时运行多个app_object对象；
kbroker分布式操作系统提供通过对象编号object_id创建app_object对象的虚接口，app_service模块通过业务层重写的所述虚接口创建app_object对象，通过app_object对象的初始化接口初始化app_object对象；
同一个app_object对象的业务逻辑在app_service模块上的同一个线程上排队进行处理，app_object对象的每一个任务事件都封装成协程进行执行，kbroker分布式操作系统提供对阻塞型接口的协程化支持；
app_service模块对app_object对象路由信息进行缓存，通过app_object对象对应的app_allocator模块获取路由信息来更新缓存，并在其他app_object对象访问时更新访问请求中来源app_object对象路由信息的缓存；
业务层对外访问基于app_object对象，系统提供访问外部app_object对象的接口，该接口由app_service模块实现；app_service模块根据缓存或从对应app_allocator模块请求获得的路由信息找到app_object对象运行所在app_service模块的进程编号program_id，发送目标为该进程编号program_id的消息使得该消息发送到app_object对象运行所在的app_service模块；
app_service模块对访问外部app_object对象的远程函数调用提供协程化支持；
app_service模块收到app_object对象待处理消息后，检查目标app_object对象的运行状况并作处理；
app_service模块在监控到其过载的情况下通知app_allocator模块不再分配新的app_object对象到其上运行；
app_service模块提供用于业务层添加异步事件和延时事件的接口，协程化的锁和协程化的信号；
app_service模块支持将正在运行的app_object对象迁移到与其同一应用程序的其他app_service模块上；
存储型应用程序的app_service模块提供针对app_object对象的主从灾备框架，与业务层重写的app_object_i虚基类中主从同步相关的接口配合实现主从灾备功能；
存储型应用程序的app_service模块提供用于主从app_object对象间传递数据的接口。
进一步地，业务层通过重写app_object_i虚基类来实现业务逻辑：
计算型应用程序的app_object_i虚基类和存储型应用程序的app_object_i虚基类都包括function_api接口、notify_api接口、初始化接口、删除接口、导出数据接口、导入数据接口；function_api接口用于处理需要返回值的访问请求，notify_api接口用于处理不需要返回值的访问请求；
业务层模块通过重写function_api接口、notify_api接口实现业务逻辑处理；业务层模块重写初始化接口用于app_service模块创建app_object对象后执行初始化；业务层模块重写删除接口用于app_service模块在关闭app_object对象时释放业务层创建的数据；导出数据接口和导入数据接口用于app_object对象的迁移，默认情况下为不支持app_object对象迁移，如果业务层重写这两个接口则该应用程序支持app_object对象的迁移，当业务负载高时将app_object对象迁移到负载低的app_service模块上；
存储型应用程序的app_service模块实现主从灾备，其app_service模块实现针对其存储型资源的数据的整体导出导入接口和主从切换接口，另外其app_object_i虚基类还包括主从同步接口，主从同步接口配合app_service模块的主从灾备框架实现业务层逻辑数据的主从灾备。
进一步地，所述网关模块包括session_allocator模块、kbroker_gateway模块；
所述session_allocator模块基于app_allocator模块的基础功能进行扩展开发，保存所述kbroker_gateway模块的socket监听信息，实现多个所述kbroker_gateway模块的负载均衡；
所述kbroker_gateway模块基于app_service模块的基础功能进行扩展开发，支持各种基于网络的外部请求访问方式，在其上运行的app_object对象为被动创建且包括对象编号object_id的外部请求连接；
所述kbroker_gateway模块用于接收外部请求，并从请求中解析出请求对应目标app_object对象的应用程序编号app_id和对象编号object_id；
所述kbroker_gateway模块支持对外部请求设定默认消息处理策略和基于请求对应目标app_object对象的应用程序编号app_id的特殊处理策略，通过外部请求中解析出来的目标app_object对象的应用程序编号app_id选择确定处理策略；如果有特殊处理策略则执行特别处理策略，否则执行默认处理策略；
所述kbroker_gateway模块还包括提供给系统内其他应用程序访问的notify_api接口、function_api接口；所述notify_api接口、function_api接口组成了系统内消息处理策略， 所述系统内消息处理策略为默认消息处理策略或基于消息来源app_object对象的应用程序编号app_id的特殊消息处理策略，支持设定针对某个应用程序编号app_id的特殊消息处理策略。
进一步地，kbroker分布式操作系统包括一单例应用程序：
所述单例应用程序管理kbroker分布式操作系统内的单例型业务逻辑，所述单例型业务逻辑为不能分配到多个地方而只在一个程序进程中执行的业务逻辑；
所述单例应用程序为kbroker分布式系统中一个特殊的应用程序；所述单例应用程序的应用程序编号为系统特殊分配的单例应用程序编号singleton_app_id；所述单例应用程序的app_service模块为各种功能不一的单例型业务逻辑的程序进程；
所述单例应用程序的app_allocator模块为每个app_service模块分配一个单例对象编号singleton_object_id，用于app_service模块相对于其他应用程序展现为单例应用程序的一个对象，业务层通过单例应用程序编号singleton_app_id和app_service模块的单例对象编号singleton_object_id访问单例应用程序的app_service模块；
所述单例应用程序的app_service模块支持主从灾备，app_allocator模块管理维护app_service模块的主从灾备；
所述单例应用程序的app_service模块包括一个唯一的业务层app_object对象，所有业务层的请求都由这个app_object对象处理；
所述单例应用程序的app_allocator模块由kbroker分布式操作系统提供，kbroker_super模块和kbroker_server模块对启动所述单例应用程序提供支持。
进一步地，kbroker_super模块支持主从灾备，当从kbroker_super模块不可用时，主kbroker_super模块启动新的从kbroker_super模块实现替代；当主kbroker_super模块不可用时，所有从kbroker_super模块选举确定其中一个从kbroker_super模块切换成主kbroker_super模块，然后再启动新的从kbroker_super模块实现替代。
进一步地，还包括管理后台，用于操作kbroker分布式操作系统：
管理后台提供访问接口给kbroker分布式操作系统，kbroker分布式操作系统通过配置或启动参数在启动时获得该访问接口；kbroker分布式操作系统通过该接口上报状态信息，状态信息包括管理网关的连接信息、服务器运行的监控信息、各个应用程序运行的状况、针对异常情况的报警、针对特定状况的提前预警；
管理后台通过管理网关获取kbroker分布式操作系统的运行信息和发送操作指令管理kbroker分布式操作系统；
管理后台在收到kbroker分布式操作系统上报的报警或预警信息时，报警通知运维和开发人员进行处理。
进一步地，在支持更大规模的服务器集群时，kbroker分布式操作系统抽出一个kbroker_messager模块进行消息转发：
kbroker_messager模块基于进程编号program_id进行消息转发；
系统中的每个kbroker_server模块都与一个kbroker_messager模块通信连接，kbroker_messager模块保存与其相连的kbroker_server模块的进程编号program_id以及在其服务器上运行的其他程序进程的进程编号program_id，将这些进程编号program_id标记为归自己管理，并将这些进程编号program_id和其与该kbroker_server模块的通信连接进行绑定；
多个kbroker_messager模块之间通信连接，kbroker_messager模块保存其他与之相连的kbroker_messager模块上管理的进程编号program_id，并将进程编号program_id和所属kbroker_messager模块与其之间的通信连接进行绑定；
kbroker_super模块管理kbroker_messager模块，分配每一个kbroker_server模块连接到一个kbroker_messager模块；
当使用kbroker_messager模块进行消息转发时，kbroker_server模块之间不进行通信连接，kbroker_server模块在收到其管理的程序进程发送的消息后转发给kbroker_messager模块。
进一步地，本发明还提供一种存储介质，其上存储有计算机程序，该程序被处理器执行时以实现如上所述的一种kbroker分布式操作系统。
进一步地，本发明还提供一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时以实现如上所述的一种kbroker分布式操作系统。
如上所述，在本发明的支持下，开发者只用关注于自身业务本身，像开发最简单的单线程程序一样开发大型服务器端程序，不需要关注随访问量增加的一系列问题。提高了服务器集群效率，提供针对软硬件故障的容灾机制以实现高可用性。
附图说明
图1显示为本发明实施例中一种kbroker分布式操作系统、存储介质和电子设备的系统结构示意图；
图2显示为本发明实施例中一种kbroker分布式操作系统、存储介质和电子设备的系统启 动流程图；
图3显示为本发明实施例中一种kbroker分布式操作系统、存储介质和电子设备的应用程序启动流程图；
图4显示为本发明实施例中一种kbroker分布式操作系统、存储介质和电子设备的启动应用程序过程中启动app_service模块的流程图；
图5显示为本发明实施例中一种kbroker分布式操作系统、存储介质和电子设备的系统结构原理图。
具体实施方式
以下通过特定的具体实例说明本发明的实施方式，本领域技术人员可由本说明书所揭露的内容轻易地了解本发明的其他优点与功效。本发明还可以通过另外不同的具体实施方式加以实施或应用，本说明书中的各项细节也可以基于不同观点与应用，在没有背离本发明的精神下进行各种修饰或改变。需说明的是，在不冲突的情况下，以下实施例及实施例中的特征可以相互组合。
需要说明的是，以下实施例中所提供的图示仅以示意方式说明本发明的基本构想，遂图式中仅显示与本发明中有关的组件而非按照实际实施时的组件数目、形状及尺寸绘制，其实际实施时各组件的型态、数量及比例可为一种随意的改变，且其组件布局型态也可能更为复杂。
如图1至图5所示，本发明提供一种kbroker分布式操作系统，kbroker分布式操作系统是一种应用于多台服务器的分布式操作系统，包括：kbroker_server模块、业务层模块、kbroker_super模块、网关模块。
每台服务器上都有且只有一个kbroker_server模块在运行，其为该服务器的代理，管理该服务器上的一切功能，包括监控和上报可用资源，负责在该服务器上启动app_allocator模块和app_service模块，并实现启动后与该服务器上的kbroker_server模块通过进程间通信连接，关闭和监控该服务器上运行的app_allocator模块和app_service模块，创建和关闭存储型资源。
kbroker_super模块通过kbroker_server模块管理所有的服务器。通过kbroker_server模块之间的互联实现将各个服务器上的资源集中成一个大的资源池，并通过kbroker_server模块根据目标程序进程的进程编号program_id进行消息转发来实现系统内所有程序之间的消息传递。kbroker_super模块通过在整个资源池里选择合适的kbroker_server模块启动新的程序和创建存储型资源来实现资源的有效利用。
业务层逻辑被分成若干个应用程序来实现，每个应用程序都由与之对应的app_allocator和app_service两种模块来组成。其中app_allocator模块用于管理应用程序的app_object对象，包括app_object对象的创建、删除和分配到具体的app_service模块上运行；app_service模块是app_object对象的对象池，负责分配到其上运行的app_object对象的业务逻辑的具体执行。当app_object对象数量或者访问量过多导致其对象池(app_service模块)不能满足业务逻辑运行需求时，app_allocator模块就向kbroker_super模块申请启动新的app_service模块。整个系统的业务逻辑由运行在app_service模块上的app_object对象的逻辑处理以及app_object对象之间的相互调用实现。
整个分布式操作系统的管理分成两层：通过kbroker_super模块和kbroker_server模块实现将所有服务器的资源整合成一个资源池来给业务层的应用程序来使用；通过app_allocator模块和app_service模块来实现对应用程序业务层app_object对象的管理。前者实现了在服务器资源不足时可以直接通过添加服务器扩充资源池来满足需求，后者实现了业务层只需要关注业务本身而不需要关心负载问题(app_object对象的数量)。
充分利用协程是本系统的一大特点，这样做的目的有两点：一是使其不阻塞线程从而可以将app_object对象的逻辑处理放在一个线程上排队执行，简化了逻辑开发的复杂度；二是实现对远程app_object对象的非阻塞调用，使得业务层类似调用本地接口一样调用远程app_object对象的接口，从而使得整个系统相对于开发者来说就像一个单线程程序。
通过系统的两层管理和协程的使用，使得整个系统对业务层来说类似一个计算资源可以随意扩展的单线程程序，不需要关心负载，只需要关心业务，业务层逻辑可以用面向对象编程思想编写类功能来简单实现。再加上系统自带的灾备机制，使得大负载服务器程序的编写和可靠运行得到极大的简化。
网关模块，为专用的业务层模块，用于接收和响应外部请求。
kbroker分布式操作系统中的应用程序包括：计算型应用程序，其业务逻辑只涉及到运算和临时在内存中保存数据；存储型应用程序，其业务逻辑涉及到使用存储型资源保存数据。存储型资源为磁盘或第三方外部程序等用于保存数据以防止分布式操作系统重启后丢失数据的存储介质。
存储型应用程序是指app_service模块，也就是业务层用到存储型资源的应用程序；kbroker_super模块和一些app_allocator模块也都用到存储型资源，这些都称之为涉及存储资源的模块。
涉及存储资源的模块需要与其用到的存储型资源运行在同一台服务器上，为此 kbroker_super模块在为涉及存储资源的模块选择kbroker_server模块时，选择其用到的存储型资源所在的kbroker_server模块而非资源上最佳的kbroker_server模块。
为了防止存储型资源失效而造成数据丢失，所有涉及存储资源的模块都需要做主从灾备。系统为主从灾备提供统一的框架支持和主从间的消息传递支持，使得业务层可以通过重写数据的导出导入接口以及数据同步的发送和处理接口就可以完成主从同步。
所有已使用的存储型资源都被设置资源编号resource_id并由kbroker_super模块保存，涉及到存储型资源的模块与存储型资源的资源编号resource_id的关联关系需要被合适的模块保存，模块之间的主从灾备关系通过保存涉及到的存储型资源的资源编号resource_id的组合关系来实现。
kbroker_server模块管理存储型资源的具体生成工作，包括一个本地文件用于生成后保存存储型资源的连接访问方式，当有程序进程用到该存储型资源时，kbroker_server模块在启动程序进程时传入该存储型资源的连接访问方式，使得程序进程连接并访问其需要的该存储型资源；kbroker_server模块还用于管理存储型资源的销毁删除工作。
kbroker_super模块包括一存储型资源，用于保存服务器信息、资源编号resource_id与kbroker_server模块的对应关系、应用程序的信息、kbroker_super模块所用到的存储型资源。服务器信息包括服务器的登录信息、服务器的编号、启动kbroker_server模块的命令。应用程序的信息包括应用程序的编号、应用程序的类型、应用程序的参数、应用程序的app_allocator模块用到的存储型资源。
kbroker_super模块维护并记录所有kbroker_server模块所在服务器上CPU、内存、磁盘、网络的可用状况，并据此为新分配的存储型资源和新启动的程序进程选择合适的kbroker_server模块。
kbroker分布式操作系统中的程序进程之间通过进程编号program_id进行通信；kbroker_server模块保存进程编号program_id和其所在服务器上的kbroker_server模块的关联关系，系统中的消息传递由kbroker_server模块转发。
kbroker_super模块通过kbroker_server模块管理其所在的服务器，kbroker_super模块通过启动和关闭服务器上的kbroker_server模块，实现向kbroker分布式操作系统内添加和删除该kbroker_server模块所在的服务器。每台kbroker分布式操作系统内使用的服务器上包括一个唯一的kbroker_server模块。kbroker_server模块监控其所在服务器的运行状况和CPU、内存、磁盘、网络的可用情况，并上报给kbroker_super模块。多个服务器之间通过在其上运行的kbroker_server模块的网络连接来实现互联，kbroker_server模块监控连接的状况并上报给 kbroker_super模块，kbroker_super模块据此得到kbroker_server模块的运行异常情况并做处理。
kbroker_super模块启动kbroker_server模块的过程中：kbroker_server模块启动后，会从配置中读取自己监听的网络端口进行监听，在向kbroker_super模块注册的过程中，也会携带自己监听的端口作为参数上传，kbroker_super模块在校验成功后会保存kbroker_server的网络地址和监听端口。
kbroker_server模块注册成功后，向kbroker_super模块请求其他正在运行的kbroker_server的网络地址和监听端口，然后根据这些信息连接到其他的kbroker_server模块，并经过校验后完成连接。
kbroker_server模块连接上一个其他正在运行的kbroker_server模块或者被其他正在运行的kbroker_server模块连接后，就向其请求其上运行的所有程序进程的进程编号program_id，获取后就将这些进程编号program_id和其kbroker_server模块的进程编号program_id关联起来，并将这些进程编号program_id的转发规则关联到自己与其kbroker_server模块的网络连接上。
kbroker_server模块在启动完成注册校验，并且已经和其他所有正在运行的其他kbroker_server模块连接以及获取到其上运行的程序后，才向kbroker_super模块通知该kbroker_server模块初始化成功，kbroker_super模块收到初始化成功的通知后，向其请求其上可用资源的状况再将其放到可使用队列中。
kbroker_super模块通过启动和关闭服务器上的kbroker_server模块来实现向kbroker分布式操作系统内添加和删除该kbroker_server模块所在的服务器，具体包括：kbroker_super模块保存所有服务器的登录信息、服务器的编号、启动kbroker_server模块的执行命令；kbroker_super模块利用服务器登录信息登录服务器；kbroker_super模块在登录的服务器上执行启动命令启动kbroker_server模块；启动后的kbroker_server模块根据启动命令中的kbroker_super模块的信息连接到kbroker_super模块，并向kbroker_super模块请求注册校验；kbroker_super模块校验注册请求，校验成功则kbroker_server模块启动成功；kbroker_super模块通过通知已校验成功的kbroker_server模块自行关闭实现从分布式操作系统中删除其所在服务器。
kbroker_super模块还用于在运行过程中向分布式操作系统中添加新服务器：向kbroker_super模块提交新服务器的登陆信息以及在新服务器上启动对应的kbroker_server模块命令，kbroker_super模块对新服务器进行编号；kbroker_super模块启动并校验新服务器对应的kbroker_server模块，如果效验成功，则添加新服务器成功，同时kbroker_super模块保 存新服务器的信息。
当应用程序启动时：kbroker_super模块根据该应用程序需要的app_allocator模块数量，为该应用程序的每个app_allocator模块选择合适的kbroker_server模块并通过kbroker_server模块启动该app_allocator模块；kbroker_super模块从已启动的该应用程序的app_allocator模块中选择确定应用程序对应的主app_allocator模块，将该应用程序其他已启动的app_allocator模块设置成从app_allocator模块并与主app_allocator模块之间完成同步，实现主从灾备，并在运行过程中维护主从灾备；主app_allocator模块启动app_service模块，并在运行过程中根据运行资源的需要新增或者缩减app_service模块的运行数量。
app_allocator模块启动app_service模块的具体过程如下：app_allocator模块向kbroker_super模块发送启动一个app_service模块的请求命令；kbroker_super模块为新启动的app_service模块生成进程编号program_id，依据启动请求选择合适的kbroker_server模块，并向该kbroker_server模块发送启动选择的app_service模块的启动指令；被选中的kbroker_server模块依据启动指令启动对应的app_service模块；app_service模块启动后通过系统进程间通信连接到kbroker_server模块，然后根据启动中的参数完成注册到kbroker_server模块的验证。kbroker_server模块验证app_service模块后向kbroker_super模块回复启动成功，但此时并不将该app_service模块添加到自己拥有的程序进程中；kbroker_super模块在收到kbroker_server模块启动app_service模块成功的回复后，保存新启动的app_service模块的进程编号program_id以及其与kbroker_server模块的对应关系，向kborker_server模块回复确认该app_service模块启动成功并携带app_allocator模块的请求参数；kbroker_server模块收到确认app_service模块启动成功的信息后，将该app_service模块添加到自己拥有的程序进程中，同时通知其他kbroker_server模块自己新增了这个程序进程。然后根据信息中的app_allocator模块的请求参数向app_allocator模块回复启动app_service模块成功并携带新启动app_service模块的进程编号program_id；app_allocator模块收到kbroker_server模块启动app_service模块成功的回复后，对该app_service模块设置参数配置以完成初始化并添加到可用队列中。
kbroker_super模块在启动用到存储型资源的app_allocator模块时：kbroker_super模块保存app_allocator模块使用的存储型资源的资源编号resource_id和app_allocator模块所属应用程序的应用程序编号app_id的关联关系；kbroker_super模块在启动其app_allocator模块用到存储型资源的应用程序时，找到该应用程序的应用程序编号app_id关联的所有存储型资源的资源编号resource_id，根据每个资源编号resource_id找到对应的kbroker_server模块并启动使 用该存储型资源的app_allocator模块，如果没有app_allocator模块启动成功则应用程序启动失败；如果有app_allocator模块启动成功则再判断启动成功的数量，如果数量满足主从灾备的需要则完成并维护主从灾备，否则先启动新的app_allocator模块满足主从灾备的数量，然后再完成并维护主从灾备；当kbroker_super模块启动一个新的app_allocator模块时，创建存储型资源，通过此存储型资源启动新的app_allocator模块，kbroker_super模块保存新创建的存储型资源的资源编号resource_id和app_allocator模块所属应用程序的应用程序编号app_id的关联关系；当使用中的app_allocator模块不可用时，删除其使用的存储型资源以及该存储型资源的资源编号resource_id和app_allocator模块所属应用程序的应用程序编号app_id的关联关系，然后启动一个新的app_allocator模块替换掉不可用的app_allocator模块。
kbroker_server模块负责系统内的消息转发：所有的消息都通过kbroker_server模块上的消息转发机制进行转发，消息转发机制是基于目标程序进程的进程编号program_id来进行；当kbroker_server模块收到一个待转发消息时，会首先找出消息目标程序进程的进程编号program_id，然后根据进程编号program_id的情况进行转发。具体情况有以下几种：进程编号program_id为当前的kbroker_server模块所拥有时，通过目标程序进程与自己的进程间通信转发过去；进程编号program_id为其他kbroker_server模块所拥有时，通过该进程编号program_id对应的kbroker_server模块与自己的网络连接转发出去；进程编号program_id不存在时，直接丢弃消息
Figure PCTCN2020112816-appb-000001
，如果短时间内没有和kbroker_super模块确认过该进程编号program_id是否存在就向kbroker_super模块请求确认该进程编号program_id是否存在；如果已经在短时间内和kbroker_super模块确认该进程编号program_id不存在，那么消息的发送方是自己拥有的情况就通知其进程编号program_id不存在，如果发送方是其他kbroker_server模块所有，就通知该对应的kbroker_server模块自己不拥有该进程编号program_id。
app_service模块由其应用程序对应的app_allocator模块负责管理，当app_allocator模块启动app_service模块时，通过kbroker_super模块选择合适的kbroker_server模块并由被选择的kbroker_server模块来完成具体的启动工作。当app_service模块不可用时，app_allocator模块删除或修改该app_service模块上运行的app_object对象在其上的路由信息缓存，根据需要决定是否启动新的app_service模块。当app_service模块过多时，app_allocator模块将空闲的app_service模块。
kbroker_server模块负责管理其所在服务器上运行的app_allocator模块和app_service模块：app_allcator模块和app_service模块由kbroker_server模块来执行启动操作，启动后和kbroker_server模块跑在同一台服务器上并通过系统进程间通信连接于kbroker_server模块； app_allocator模块和app_service模块与kbroker分布式操作系统内其他程序进程的消息传递均由kbroker_server模块进行转发；kbroker_server模块监控app_allocator模块和app_service模块的异常状况，上报kbroker_super模块并处理；kbroker_server模块根据其所在服务器CPU、内存、磁盘、网络的总体使用情况，对其上运行的app_allocator模块和app_service模块进行过载设置、迁移、关闭的管理操作。
kbroker_server模块根据自身服务器总体资源使用情况，对其上运行的app_allocator模块和app_service模块进行过载设置、迁移、关闭等管理操作。kbroker_server模块监控到自身服务器已使用的资源达到设定的可用阈值时，将其上运行的所有app_service模块设置成过载状态，使其通知其对应的app_allocator模块不再对其分配新的app_object对象。kbroker_server模块监控到自身服务器已使用的资源达到设定的极限阈值时，开启迁移工作。
首先通知计算型app_service模块缩短app_object对象的移除时间间隔，从一个相对长时间没有访问才移除变更为短时间内没有访问请求就移除app_object对象并通知app_allocator模块移除该app_object对象的路由。
如果操作后资源使用量还过高，接下来就通知计算型app_service模块迁移app_object对象，如果app_object对象支持迁移(实现了导出导入接口)，那么会将其迁移到其他空闲的app_service模块上。
凡是涉及到存储型资源的都会有主从灾备模式，在上述操作后使用资源仍过高的极端情况下直接关闭其上运行的主从灾备模式下的从app_allocator模块和从app_service模块。
如果关闭从服务后资源使用量还过高，那就将主从灾备模式下的主app_allocator模块和主app_service模块切换主到其他从模块上而使自身转换为从，之后再实施关闭操作。
当app_allocator模块管理app_object对象时：app_allocator模块主要功能就是管理app_object对象，根据正在运行的app_object对象的情况启动足够的app_service模块，并把这些需要运行的app_object对象分配到合适的app_service模块上去运行，同时对系统内提供查询app_object对象路由信息(具体运行在哪个app_service模块上)的接口。
由于app_service模块会缓存路由信息，所以当app_allocator模块对一个路由信息请求回复该app_object对象不存在时，需要记录曾向请求方返回对象不存在，以备接下来该app_object对象被创建时通知请求方更新其针对该app_object对象的路由缓存。
计算型应用程序的app_object对象分为永久存在和临时存在两种模式：永久存在型：这种对象就是创建了就一直存在，除非再将其删除；没有创建时就是不存在的。比如常见的用户注册功能，注册了该用户就存在，没有注册就不存在。临时存在型：这种用于外界并不关心 或者确认具体对象，由app_allocator模块根据需要来自行管理创建和销毁的情况。比如不加限制的多人拼团买东西，用户只关心是否拼团成功而不关心拼的是具体哪个团，对于拼团这个应用程序的app_allocator模块，其自行管理每一个拼团，并将申请参加拼团的用户分配到合适的拼团上。
当某一个app_service模块崩溃时，app_allocator模块会删除该app_service模块上运行的所有app_object对象在其上保存的路由信息，等接下来遇到针对这些app_object对象的路由信息请求时再把对应的app_object对象分配到新的app_service模块上运行。另外会根据需要考虑是否启动新的app_service模块。
app_allocator模块还用于创建管理应用程序的app_object对象：计算型应用程序的app_object对象的模式包括永久存在模式和临时存在模式，存储型应用程序的app_object对象的模式为永久存在模式；app_allocator模块通过app_service模块执行创建启动app_object对象；app_allocator模块保存app_object对象的路由信息，路由信息为app_object对象运行在哪个app_service模块上，并对外提供请求路由信息的接口；app_allocator模块对于永久存在模式的app_object对象，提供相应的接口函数对其进行创建和删除；永久存在模式的app_object对象当被访问时启动运行，长时间没有访问请求的情况下app_object对象将被关闭并退出运行；对于管理永久存在模式app_object对象的app_allocator模块，包括一个存储型资源用于对已存在的app_object对象进行保存；对于临时存在模式的app_object对象，由app_allocator模块根据需要自行分配；app_allocator模块通过添加专属功能满足应用程序对app_object模块的特定要求。
存储型应用程序的app_allocator模块维护app_service模块的主从灾备，存储型应用程序的app_allocator模块把若干个相应的app_service模块编成一组，并对每个分组设置分组编号virtual_id，维护每个分组内app_service模块的主从灾备。
存储型应用程序的app_service模块都设有与之对应的存储型资源，存储型应用程序的app_allocator模块保存分组的分组编号virtual_id与该分组内app_service模块用到的存储型资源的资源编号resource_id的关联关系。
已存在的app_object对象与分组一一对应绑定，存储型应用程序的app_allocator模块保存该对应关系；新的app_object对象在申请创建时选择一个分组进行绑定，当没有可用的分组时先创建新的分组和分组内的app_service模块，然后再选择一个分组进行绑定。
存储型应用程序的app_allocator模块在处理对app_object对象的路由信息请求时，查找到该app_object对象对应分组的分组编号virtual_id后，找到该分组编号virtual_id对应分组 内的主app_service模块的进程编号program_id返回给请求方。
app_object对象的具体逻辑：app_object对象由应用程序的应用程序编号app_id以及app_object对象在应用程序上的对象编号object_id来唯一标识，业务层通过应用程序的应用程序编号app_id以及app_object对象在应用程序上的对象编号object_id对app_object对象进行访问；业务层通过重写app_service模块提供的app_object_i虚基类来实现业务逻辑；单个app_object对象的逻辑处理只能运行在单个app_service模块上，单个app_service模块上同时运行多个app_object对象。
kbroker分布式操作系统提供通过对象编号object_id创建app_object对象的虚接口，app_service模块通过业务层重写的所述虚接口创建app_object对象，通过app_object对象的初始化接口初始化app_object对象。
同一个app_object对象的业务逻辑在app_service模块上的同一个线程上排队进行处理，app_object对象的每一个任务事件都封装成协程进行执行，kbroker分布式操作系统提供对阻塞型接口的协程化支持，支持的方法是通过hook机制替换操作系统的阻塞型接口为重写后支持协程的非阻塞接口。
app_service模块对app_object对象路由信息进行缓存，通过app_object对象对应的app_allocator模块获取路由信息来更新缓存，并在其他app_object对象访问时更新访问请求中来源app_object对象路由信息的缓存。
业务层对外访问基于app_object对象，系统提供访问外部app_object对象的接口，该接口由app_service模块实现；app_service模块根据缓存或从对应app_allocator模块请求获得的路由信息找到app_object对象运行所在app_service模块的进程编号program_id，发送目标为该进程编号program_id的消息使得该消息发送到app_object对象运行所在的app_service模块。
app_service模块对访问外部app_object对象的远程函数调用提供协程化支持。
app_service模块收到app_object对象待处理消息后，检查目标app_object对象的运行状况并作处理，app_service模块在监控到其过载的情况下通知app_allocator模块不再分配新的app_object对象到其上运行。
app_service模块提供用于业务层添加异步事件和延时事件的接口，协程化的锁和协程化的信号。
app_service模块支持将正在运行的app_object对象迁移到与其同一应用程序的其他app_service模块上。
存储型应用程序的app_service模块提供针对app_object对象的主从灾备框架，与业务层 重写的app_object_i虚基类中主从同步相关的接口配合实现主从灾备功能，存储型应用程序的app_service模块提供用于主从app_object对象间传递数据的接口。
app_service模块是实现具体业务逻辑的模块，系统的其他模块都是为了为其提供支撑，使得整个系统可以运行大量各种应用程序的app_service模块，进而支持大量的app_object对象同时运行，将整个分布式系统相对于开发者而言变成一个简单的单线程程序。
app_service模块就是app_object对象的对象池，通过业务层重写的创建对象接口来创建对象，并将发送到该模块的某个app_object对象的消息传递给其上运行的对应app_object对象来执行。
app_service模块收到业务层消息时，会首先判断目标app_object对象的状况：目标app_object对象处于正常状态，直接调用app_object对象的接口执行；目标app_object对象正在初始化，则缓存消息，待初始化完成后再依次执行；目标app_object对象正在执行迁移，则缓存消息，后期若迁移成功就依次投递到其迁移后所在的app_service模块上执行；若迁移失败则app_obect对象恢复至正常状态，缓存的消息再依次执行；目标app_object对象不在自己上面，则查找目标app_object对象的路由信息并转发消息。
系统为业务层提供用于访问远程对象的notify_api_call接口和function_api_call接口，其都是通过目标app_object对象的应用程序编号app_id和对象编号object_id做参数来访问远程对象。notify_api_call接口不需要返回值只用投递消息就可以了；function_api_call接口是协程化的远程函数调用，在投递完消息后保存堆栈让出CPU，然后等远程app_object对象返回消息后再恢复堆栈拿到返回值继续执行。
整个分布式系统是为了开发者可以像开发最简单的单线程程序一样开发大型分布式服务器端程序，所以很多地方都为此做了适应常规思维的处理，将app_object对象的类和传统编程中的类相对应：app_object对象的function_api接口对应于类中需要返回值函数的总入口；app_object对象的notify_api接口对应于类中不需要返回值函数的总入口；notify_api_call接口将远程调用对象不需要返回值的函数接口转换为消息投递，function_api_call接口将远程调用对象需要返回值的函数接口协程化，业务层可以像使用本地调用一样使用远程调用。
业务层应用程序利用notify_api_call接口和function_api_call接口将对象对外提供的接口调用封装成一个头文件，其他业务层直接使用这些封装后的函数接口来调用对象的函数接口。这个封装后的头文件就类似传统程序中类的头文件。为了让开发者可以使用现有的设计模式，我们提供了针对当前app_object对象的单例——object_singleton，开发者可以继承object_singleton来像传统的单例模式一样使用。
app_service模块在实现添加异步事件和延时事件接口的功能时，不能在业务层调用接口时直接添加，而应该缓存这些事件，等当前任务逻辑执行完成后再添加，以免出现当前逻辑还没有执行完添加的异步事件或者延时事件就执行的问题。app_service模块为配合协程化信号还另外提供一个直接添加异步事件的接口。
app_service模块提供的协程化锁会在有其他任务已经加锁的情况下缓存堆栈，让出CPU，等其他任务释放锁之后再恢复堆栈继续执行，另外还能通过返回值判断是否是被中断后又恢复执行的，以及通过错误参数判断是否是超时退出。
app_service模块提供的协程化信号，主要有监听和发送两个功能，在监听时缓存堆栈让出CPU，等其他任务发送对应信号后恢复堆栈继续执行。
业务层通过重写app_object_i虚基类来实现业务逻辑，计算型应用程序的app_object_i虚基类和存储型应用程序的app_object_i虚基类都包括function_api接口、notify_api接口、初始化接口、删除接口、导出数据接口、导入数据接口；function_api接口用于处理需要返回值的访问请求，notify_api接口用于处理不需要返回值的访问请求。
业务层模块通过重写function_api接口、notify_api接口实现业务逻辑处理；业务层模块重写初始化接口用于app_service模块创建app_object对象后执行初始化；业务层模块重写删除接口用于app_service模块在关闭app_object对象时释放业务层创建的数据；导出数据接口和导入数据接口用于app_object对象的迁移，默认情况下为不支持app_object对象迁移，如果业务层重写这两个接口则该应用程序支持app_object对象的迁移，当业务负载高时将app_object对象迁移到负载低的app_service模块上。
存储型应用程序的app_service模块实现主从灾备，其app_service模块实现针对其存储型资源的数据的整体导出导入接口和主从切换接口，另外其app_object_i虚基类还包括主从同步接口，主从同步接口配合app_service模块的主从灾备框架实现业务层逻辑数据的主从灾备。
网关模块包括session_allocator模块、kbroker_gateway模块；session_allocator模块基于app_allocator模块的基础功能进行扩展开发，保存所述kbroker_gateway模块的socket监听信息，实现多个所述kbroker_gateway模块的负载均衡。
kbroker_gateway模块基于app_service模块的基础功能进行扩展开发，支持各种基于网络的外部请求访问方式，在其上运行的app_object对象为被动创建且包括对象编号object_id的外部请求连接。
kbroker_gateway模块用于接收外部请求，并从请求中解析出请求对应目标app_object对象的应用程序编号app_id和对象编号object_id。kbroker_gateway模块支持对外部请求设定默 认消息处理策略和基于请求对应目标app_object对象的应用程序编号app_id的特殊处理策略，通过外部请求中解析出来的目标app_object对象的应用程序编号app_id选择确定处理策略。如果有特殊处理策略则执行特别处理策略，否则执行默认处理策略。
kbroker_gateway模块还包括提供给系统内其他应用程序访问的notify_api接口、function_api接口；notify_api接口、function_api接口组成了系统内消息处理策略，系统内消息处理策略为默认消息处理策略或基于消息来源app_object对象的应用程序编号app_id的特殊消息处理策略，支持设定针对某个应用程序编号app_id的特殊消息处理策略。
网关模块主要是接收外部的请求，并从请求中解析出目标app_object对象，然后将请求发送到目标app_object对象去处理，具体操作上也支持对请求的特殊处理逻辑。从网关的类型上可以分为长连接型和短连接型。
长连接型请求通常需要登录验证，登录验证的工作就由session_allocator模块来负责处理，验证成功后就有了内部object_id，这样直接把连接和object_id绑定就可以了，之后的操作就和常规的app_object类似了。也有长连接型请求是不需要验证的，这种情况可以视为将多个短连接合并到一个连接里面，处理方法类似下面的短连接型请求。
短连接型请求最典型的就是http请求，我们以http请求来说明，http请求类似一个函数式调用，一个请求一个返回，我们从请求中解析出目标app_object对象，保存socket连接，然后直接利用系统提供的function_api_call接口去访问目标app_object对象，函数返回后将返回数据发送给socket连接即可。
整个分布式系统的网关在功能类型上可以分成业务层网关、系统管理网关、业务管理网关。
业务层网关是业务层为了接收外部请求而提供的网关，用于实现系统业务层对外提供的访问接口。
系统管理网关是提供给系统管理人员使用的网关接口，系统管理人员通过这些网关接口获得当前系统的运行情况和故障报警等信息，也可以通过这些接口执行一些操作(包括但不限于增删服务器，强制关闭某个应用程序等)来维护系统的运行。
业务管理网关是提交给开发人员使用的网关，功能包括查询应用程序的运行状况和报警信息等，增删应用程序，发布应用程序等功能。
kbroker分布式操作系统包括一单例应用程序，单例应用程序管理kbroker分布式操作系统内的单例型业务逻辑，所述单例型业务逻辑为不能分配到多个地方而只在一个程序进程中执行的业务逻辑。
单例应用程序为kbroker分布式系统中一个特殊的应用程序，单例应用程序的应用程序编号为系统特殊分配的单例应用程序编号singleton_app_id，单例应用程序的app_service模块为各种功能不一的单例型业务逻辑的程序进程。
单例应用程序的app_allocator模块为每个app_service模块分配一个单例对象编号singleton_object_id，用于app_service模块相对于其他应用程序展现为单例应用程序的一个对象，业务层通过单例应用程序编号singleton_app_id和app_service模块的单例对象编号singleton_object_id访问单例应用程序的app_service模块。
单例应用程序的app_service模块支持主从灾备，app_allocator模块管理维护app_service模块的主从灾备。单例应用程序的app_service模块包括一个唯一的业务层app_object对象，所有业务层的请求都由这个app_object对象处理。单例应用程序的app_allocator模块由kbroker分布式操作系统提供，kbroker_super模块和kbroker_server模块对启动所述单例应用程序提供支持。
单例应用程序是为了解决那些只能全局唯一的逻辑，比如常见的id生成器，id生成器为整个系统提供id编号的生成工作，整个系统只能有一个这样的程序来运行逻辑，这样的功能需求的实现就和前面普通应用程序分配到多个app_service模块上执行逻辑运行有很大区别，为此需要专门提供一类特殊的应用程序来支持这种功能需求。
kbroker_super模块支持主从灾备，kbroker_super模块实现一主多从的主从灾备机制。当从kbroker_super模块不可用时，主kbroker_super模块启动新的从kbroker_super模块实现替代；当主kbroker_super模块不可用时，所有从kbroker_super模块选举确定其中一个从kbroker_super模块切换成主kbroker_super模块，然后再启动新的从kbroker_super模块实现替代。
kbroker分布式系统启动主kbroker_super模块，由主kbroker_super模块启动从kbroker_super模块，并完成数据同步实现主从灾备。
主kbroker_super模块为所有的kbroker_super模块设置不同的优先级，当主kbroker_super模块失效时，由剩余可用的从kbroker_super模块选出当前可用的最高优先级的kbroker_super模块为主kbroker_super模块。
当一个从kbroker_super模块切换成主kbroker_super模块时，首先获取所有kbroker_server模块的可用资源状况和其拥有程序进程的进程编号program_id，更新kbroker_server的可用资源队列，校验并处理其缓存的kbroker_server模块拥有程序与从kbroker_server模块实际获取到数据的差异。
以上kbroker_super模块的主从灾备是不依赖于其他现有程序或者类库独立实现的，也可由zookeeper之类的开源软件辅助实现。
kbroker分布式操作系统还包括管理后台，用于操作kbroker分布式操作系统。管理后台提供访问接口给kbroker分布式操作系统，kbroker分布式操作系统通过配置或启动参数在启动时获得该访问接口；kbroker分布式操作系统通过该接口上报状态信息，状态信息包括管理网关的连接信息、服务器运行的监控信息、各个应用程序运行的状况、针对异常情况的报警、针对特定状况的提前预警。
管理后台通过管理网关获取kbroker分布式操作系统的运行信息和发送操作指令管理kbroker分布式操作系统。
管理后台在收到kbroker分布式操作系统上报的报警或预警信息时，报警通知运维和开发人员进行处理。
在支持更大规模的服务器集群时，kbroker分布式操作系统抽出一个kbroker_messager模块进行消息转发，kbroker_messager模块基于进程编号program_id进行消息转发。
系统中的每个kbroker_server模块都与一个kbroker_messager模块通信连接，kbroker_messager模块保存与其相连的kbroker_server模块的进程编号program_id以及在其服务器上运行的其他程序进程的进程编号program_id，将这些进程编号program_id标记为归自己管理，并将这些进程编号program_id和其与该kbroker_server模块的通信连接进行绑定。
多个kbroker_messager模块之间通信连接，kbroker_messager模块保存其他与之相连的kbroker_messager模块上管理的进程编号program_id，并将进程编号program_id和所属kbroker_messager模块与其之间的通信连接进行绑定。
kbroker_super模块管理kbroker_messager模块，分配每一个kbroker_server模块连接到一个kbroker_messager模块。
当使用kbroker_messager模块进行消息转发时，kbroker_server模块之间不进行通信连接，kbroker_server模块在收到其管理的程序进程发送的消息后转发给kbroker_messager模块。
考虑到将kbroker_super模块做成一个独立的程序会导致管理维护复杂，以及kbroker_super模块需要和每个kbroker_server模块进行网络连接而影响性能，为此将kbroker_super模块和kbroker_server模块合并到一个kbroker程序里，当kbroker程序所在的服务器上有kbroker_super模块时就启动程序内的kbroker_super模块，没有就不启动。这样整个分布式系统上就只需要一个关键的kbroker程序。对于应用程序的的开发支持方面，分布式系统提供app_allocator模块和app_service模块这两种模块的库文件和接口文件，应用程序通 过重写这些接口，然后和库文件编译链接后形成应用程序真正的app_allocator程序和app_service程序。
kbroker_super模块拥有一个存储型资源来保存所有服务器的登陆信息、kbroker_server编号和启动kbroker_server模块的命令；另外保存有已使用的应用程序的应用程序编号app_id；还保存有所有资源的资源编号resource_id及其所在服务器上的kbroker_server模块的关联关系。kbroker_super模块是kbroker程序的一部分，跑在kbroker程序独立的一组线程上。kbroker_super模块主要功能如下：
根据保存的服务器信息，启动所有服务器上的kbroker程序，完成kbroker程序中kbroker_server模块的验证注册，验证注册后会保存kbroker_server模块的socket监听信息。
保存维护所有kbroker_server模块的socket监听信息，并传递该信息给需要的kbroker_server模块使其能够连接到其他kbroker_server模块，从而实现所有kbroker_server模块之间的网络连接。
保存并及时更新所有kbroker_server模块上报的其所在服务器的资源情况，根据资源情况在启动应用程序的程序进程和分配资源时选择合适的服务器。
为每个运行的程序，包括kbroker_server模块所在的kbroker程序、应用程序的app_allocator程序和app_service程序，另外还包括kbroker_super模块自己，分配进程编号program_id。同时用kbroker程序进程的进程编号program_id作为服务器的编号，保存所有进程编号program_id和其所在服务器编号的对应关系。
为每个已申请的应用程序分配应用程序编号app_id并记录，启动应用程序后会保存应用程序app_allocator程序和app_service程序进程的进程编号program_id与应用程序编号app_id的对应关系，同时管理维护每个应用程序的app_allocator程序的主从灾备。
kbroker程序中kbroker_server模块主要是连接系统内的其他服务器，管理自己服务器上的程序。核心功能如下：
监听一个网络端口用于接受其他服务器上kbroker程序的连接。
kbroker程序连接到其他kbroker程序时，会发送自己的编号和校验码给对方，对方会拿这个信息以及源kbroker程序的网络地址去kbroker_super模块做校验，检验成功后双方连接成功，连接成功后双方会互相获取对方所管理的进程编号program_id并将这些进程编号program_id的消息投递与双方的网络连接做绑定。
kbroker_server模块会保存所有的进程编号program_id，以及这些进程编号program_id和其所属kbroker程序进程的进程编号program_id的对应关系，在运行过程中遇到未知进程编 号program_id或者进程编号program_id与kbroker程序进程的进程编号program_id的对应关系出现冲突时由kbroker_super模块来做裁定并处理。
kbroker_server模块会在收到kbroker_super模块的启动程序的命令后启动对应的程序，在启动某个应用程序的app_allocator模块或者app_service模块的程序时，首先会检查对应应用程序的执行文件是否存在，存在就直接启动执行文件；不存在的话就从网络上下载该应用程序对应的软件包，软件包中包含该应用程序所有的执行文件和配置，下载后解压缩，然后再启动执行文件。程序启动后通过操作系统的本地socket连接到kbroker程序，kbroker程序的kbroker_server模块在kbroker_super模块确认程序启动成功后保存新启动程序进程的进程编号program_id和其socket连接的对应关系，并将自己拥有的这个新的进程编号program_id广播给其他kbroker程序的kbroker_server模块。
kbroker程序有一个线程专门负责消息转发，所有消息根据目标进程编号program_id进行转发，如果进程编号program_id在其他的服务器上就通过socket转发到对应服务器的kbroker_server模块，如果进程编号program_id是在自己服务器上的，那就直接找到对应的socket发送到对应的程序。
kbroker程序负责监控其所在服务器的资源情况以及其上程序进程的状态，管理维护与其他kbroker程序的socket连接状况。
kbroker程序用python之类的脚本语言实现一些相对繁琐的非核心功能，比如app_allocator和app_service程序的启动、服务器的资源监控等。
app_allocator模块提供一个app_allocator类库来给业务层来使用，提供一些接口给业务层实现自己的专有逻辑，然后将专有代码和app_allocator类库合起来编译出应用程序专有的app_allocator程序。
app_allocator模块类库的核心功能是管理分配app_object对象，以及管理具体运行app_object对象的app_service模块，这些功能都已在前面进行过介绍，具体实现上注意同一个请求——比如路由请求——前后连着多次发送的情况，这种情况下可以用加锁机制避免重复操作。
app_allocator模块类库对外部传入的路由查询请求、app_service模块启动成功以及关闭、app_object对象启动成功或者关闭等核心操作提供处理接口，业务层可以通过重写这些接口来实现自己的专有逻辑，还可以指定在执行完自己的专有逻辑后是否执行类库提供的常规逻辑。
app_allocator模块总体上来说类型并不是特别多，可以在提供基础类库的基础上，再适当封装一些专有程序，比如针对常见永久性对象、mysql、文件存储、场景进入等的专用 app_allocator程序以简化开发。
app_service模块提供一个app_service类库供业务层使用，业务层通过重写app_object_i这个虚基类来实现业务层的具体功能，并重写系统提供的创建app_object对象的虚接口来实现创建真正的业务层对象。
app_service模块的业务层不涉及任何与服务器或者进程编号program_id相关的内容，只涉及到app_object对象，app_object对象通过应用程序编号app_id和对象编号object_id进行访问。
notify_api和function_api这两个接口的参数中都有cmd命令字和msg消息内容。
考虑到notify_api接口虽然不需要返回值，但有可能会在执行中对源app_object对象发送消息，因此notify_api接口的参数中需要有消息的源app_object对象的应用程序编号app_id和对象编号object_id作为参数。
function_api接口原则上应当对所有的请求来源执行相同的逻辑，所以不需要区分请求来源是哪个具体的app_object对象，但参数中可以有消息的源app_object对象的应用程序编号app_id，应用程序编号app_id是为了做访问权限控制。
notify_api_call接口、function_api_call接口、协程化的锁和信号、添加异步事件和延时事件等接口都统一到一个user_api的头文件中，业务层在需要时引用头文件并使用相关接口。
app_service模块类库在提供基础类库的基础上，也可以再针对一些比如mysql等常用需求提供专有的类库以简化开发，所有这些类库都会提供各自的设置接口用于业务层设置其重写的创建业务层对象的接口和其他参数。
网关模块用于监控网络连接、接收外部请求并转发给内部对应的app_object对象处理、将返回内容发送给请求方。同样通过提供一个kbroker_gateway模块类库来供业务层使用。
kbroker_gateway模块类库提供一个网络监听接口用于业务层设置网络监听端口。kbroker_gateway模块类库提供网络处理策略的虚基类，虚基类里包括新连接到来，收到消息，连接断开等事件处理策略的接口；并提供一个设置策略的接口。业务层重写这个虚基类实现专有逻辑，并通过设置策略的接口将重写后的策略设置上去。
kbroker_gateway模块类库提供设置客户端消息默认处理策略的接口，以及基于消息目标app_object对象的应用程序编号app_id的特殊处理策略的接口。
kbroker_gateway模块类库提供处理系统内来自其他程序的消息的专有处理策略的设置接口，该策略基于消息来源app_object对象的应用程序编号app_id，其处理策略中的具体接口是和app_object对象的notify_api接口和function_api接口一样。没有专有处理策略的情况下 使用默认策略，默认策略中notify_api接口的处理是通过目标app_object对象的网络连接发送给客户端，function_api接口的处理是直接返回出错。
一个应用程序最终会有其专用的app_allocator程序和app_service程序，并且会有其配置文件，将其打成一个压缩包，其中包括上述文件以及一个总的配置文件，这个总的配置文件会指明app_allocator程序和app_service程序的启动命令，这个压缩包会上传到一个内部服务器上供整个系统下载，每台服务器上的kbroker程序可以在需要时通过应用程序的应用程序编号app_id来下载其对应的程序包，然后解压缩到本地后启动该应用程序内的程序。
kbroker_super模块、app_allocator模块和存储型应用程序的app_service模块都有主从灾备，以上描述中没有明确指定主从的情况下默认为相应模块的主模块，只有主模块会负责逻辑处理，从模块不参与逻辑处理只负责同步保存主模块的状态。
本发明还提供一种存储介质，其上存储有计算机程序，该程序被处理器执行时以实现上述的一种kbroker分布式操作系统。
本发明还提供了一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，处理器程序进程时以实现上述的一种kbroker分布式操作系统。
综上所述，本发明的一种kbroker分布式操作系统、存储介质和电子设备，开发者只用关注于自身业务本身，像开发最简单的单线程程序一样开发大型服务器端程序，不需要关注随访问量增加而导致的一系列问题。提高了服务器集群效率，提供针对软硬件故障的容灾机制以实现高可用性。所以，本发明有效克服了现有技术中的种种缺点而具有高度产业利用价值。
上述实施例仅例示性说明本发明的原理及其功效，而非用于限制本发明。任何熟悉此技术的人士皆可在不违背本发明的精神及范畴下，对上述实施例进行修饰或改变。因此，举凡所属技术领域中具有通常知识者在未脱离本发明所揭示的精神与技术思想下所完成的一切等效修饰或改变，仍应由本发明的权利要求所涵盖。

Claims (24)

  1. 一种kbroker分布式操作系统，其特征在于，包括：
    kbroker_server模块，用于管理分布式操作系统中其所在的服务器，kbroker_server模块管理其所在服务器上的app_service模块和app_allocator模块、存储型资源；多个kbroker_server模块之间数据通信连接；
    业务层模块，用于实现整个分布式操作系统的业务逻辑；所述业务逻辑被拆分成若干个应用程序；每个应用程序包括与之对应的一组app_allocator模块和至少一组app_service模块，app_service模块用于运行所述业务逻辑，app_allocator模块用于管理应用程序的运行资源，所述运行资源为该应用程序执行其业务逻辑所需要的服务器资源；app_allocator模块通过所述运行资源的需求量分配运行应用程序的app_service模块；所述业务逻辑由运行在app_service模块上的业务层app_object对象的逻辑处理以及app_object对象之间的相互调用实现；
    kbroker_super模块，用于通过kbroker_server模块管理kbroker_server模块所在的服务器；通过kbroker_server模块管理分布式操作系统中的所有程序进程，并为每个程序进程设置进程编号program_id；通过kbroker_server模块管理分布式系统上所有已分配使用的和可用的存储型资源，并为每个已分配使用的存储型资源设置资源编号resource_id以及保存资源编号resource_id与所属kbroker_server模块的对应关系；管理业务层应用程序，并为每个业务层应用程序设置应用程序编号app_id；
    网关模块，为专用的业务层模块，用于接收和响应外部请求。
  2. 根据权利要求1所述的一种kbroker分布式操作系统，其特征在于，分布式操作系统中的应用程序包括：
    计算型应用程序，其业务逻辑只涉及到运算和临时在内存中保存数据；
    存储型应用程序，其业务逻辑涉及到使用存储型资源保存数据。
  3. 根据权利要求1所述的一种kbroker分布式操作系统，其特征在于，存储型资源为用于保存数据防止kbroker分布式操作系统重启后丢失数据的存储介质；
    kbroker_server模块管理存储型资源的具体生成工作，包括一个本地文件用于存储型资源生成后保存其连接访问方式，当有程序进程用到该存储型资源时，kbroker_server模块在启动程序进程时传入该存储型资源的连接访问方式，使得程序进程连接并访问其需要的该存储型资源；kbroker_server模块还用于管理存储型资源的销毁删除工作。
  4. 根据权利要求1所述的一种kbroker分布式操作系统，其特征在于，kbroker_super模块包括一存储型资源，用于保存服务器信息、资源编号resource_id与kbroker_server模块的对应关系、应用程序的信息、kbroker_super模块所用到的存储型资源。
  5. 根据权利要求1所述的一种kbroker分布式操作系统，其特征在于，kbroker_super模块维护并记录所有kbroker_server模块所在服务器上CPU、内存、磁盘、网络的可用状况，并据此为新分配的存储型资源和新启动的程序进程选择kbroker_server模块。
  6. 根据权利要求1所述的一种kbroker分布式操作系统，其特征在于，kbroker分布式操作系统中的程序进程之间通过进程编号program_id进行通信；kbroker_server模块保存进程编号program_id和其所在服务器上的kbroker_server模块的关联关系，系统中的消息传递由kbroker_server模块转发。
  7. 根据权利要求1所述的一种kbroker分布式操作系统，其特征在于，kbroker_super模块通过kbroker_server模块管理其所在的服务器：
    kbroker_super模块通过启动和关闭服务器上的kbroker_server模块，实现向kbroker分布式操作系统内添加和删除该kbroker_server模块所在的服务器；
    每台kbroker分布式操作系统内使用的服务器上包括一个唯一的kbroker_server模块；
    kbroker_server模块监控其所在服务器的运行状况和CPU、内存、磁盘、网络的可用情况，并上报给kbroker_super模块；
    多个服务器之间通过在其上运行的kbroker_server模块的网络连接来实现互联，kbroker_server模块监控连接的状况并上报给kbroker_super模块，kbroker_super模块据此得到kbroker_server模块的运行异常情况并做处理。
  8. 根据权利要求7所述的一种kbroker分布式操作系统，其特征在于，kbroker_super模块通过启动和关闭服务器上的kbroker_server模块来实现向kbroker分布式操作系统内添加和删除该kbroker_server模块所在的服务器，具体包括：
    kbroker_super模块保存所有服务器的登录信息、服务器的编号、启动kbroker_server模块的执行命令；
    kbroker_super模块利用服务器登录信息登录服务器；
    kbroker_super模块在登录的服务器上执行启动命令启动kbroker_server模块；
    启动后的kbroker_server模块根据启动命令中的kbroker_super模块的信息连接到kbroker_super模块，并向kbroker_super模块请求注册校验；
    kbroker_super模块校验注册请求，校验成功则kbroker_server模块启动成功；
    kbroker_super模块通过通知已校验成功的kbroker_server模块自行关闭实现从分布式操作系统中删除其所在服务器。
  9. 根据权利要求8所述的一种kbroker分布式操作系统，其特征在于，kbroker_super模块还用于在运行过程中向分布式操作系统中添加新服务器：
    向kbroker_super模块提交新服务器的登陆信息以及在新服务器上启动对应的kbroker_server模块命令；
    kbroker_super模块对新服务器进行编号；
    kbroker_super模块启动并校验新服务器对应的kbroker_server模块，如果效验成功，则添加新服务器成功，同时kbroker_super模块保存新服务器的信息。
  10. 根据权利要求1所述的一种kbroker分布式操作系统，其特征在于，当应用程序启动时：
    kbroker_super模块根据该应用程序需要的app_allocator模块数量，为该应用程序的每个app_allocator模块选择kbroker_server模块并通过kbroker_server模块启动该app_allocator模块；
    kbroker_super模块从已启动的该应用程序的app_allocator模块中选择确定应用程序对应的主app_allocator模块，将该应用程序其他已启动的app_allocator模块设置成从app_allocator模块并与主app_allocator模块之间完成同步，实现主从灾备，并在运行过程中维护主从灾备；
    主app_allocator模块启动app_service模块，并在运行过程中根据运行资源的需要新增或者缩减app_service模块的运行数量。
  11. 根据权利要求10所述的一种kbroker分布式操作系统，其特征在于，kbroker_super模块在启动用到存储型资源的app_allocator模块时：
    kbroker_super模块保存app_allocator模块使用的存储型资源的资源编号resource_id和app_allocator模块所属应用程序的应用程序编号app_id的关联关系；
    kbroker_super模块在启动其app_allocator模块用到存储型资源的应用程序时，找到该应用程序的应用程序编号app_id关联的所有存储型资源的资源编号resource_id，根据每个资源 编号resource_id找到对应的kbroker_server模块并启动使用该存储型资源的app_allocator模块，如果没有app_allocator模块启动成功则应用程序启动失败；如果有app_allocator模块启动成功则再判断启动成功的数量，如果数量满足主从灾备的需要则完成并维护主从灾备，否则先启动新的app_allocator模块满足主从灾备的数量，然后再完成并维护主从灾备；
    当kbroker_super模块启动一个新的app_allocator模块时，创建存储型资源，通过此存储型资源启动新的app_allocator模块，kbroker_super模块保存新创建的存储型资源的资源编号resource_id和app_allocator模块所属应用程序的应用程序编号app_id的关联关系；
    当使用中的app_allocator模块不可用时，删除其使用的存储型资源以及该存储型资源的资源编号resource_id和app_allocator模块所属应用程序的应用程序编号app_id的关联关系，然后启动一个新的app_allocator模块替换掉不可用的app_allocator模块。
  12. 根据权利要求10所述的一种kbroker分布式操作系统，其特征在于，app_service模块由其应用程序对应的app_allocator模块管理：
    当app_allocator模块启动app_service模块时，通过kbroker_super模块选择kbroker_server模块并由被选择的kbroker_server模块来完成具体的启动工作；
    当app_service模块不可用时，app_allocator模块删除或修改该app_service模块上运行的app_object对象在其上的路由信息缓存，根据需要决定是否启动新的app_service模块；
    当app_service模块过多时，app_allocator模块将关闭空闲的app_service模块。
  13. 根据权利要求1所述的一种kbroker分布式操作系统，其特征在于，kbroker_server模块负责管理其所在服务器上运行的app_allocator模块和app_service模块：
    app_allcator模块和app_service模块由kbroker_server模块来执行启动操作，启动后和kbroker_server模块跑在同一台服务器上并通过系统进程间通信连接于kbroker_server模块；
    app_allocator模块和app_service模块与kbroker分布式操作系统内其他程序进程的消息传递均由kbroker_server模块进行转发；
    kbroker_server模块监控app_allocator模块和app_service模块的异常状况，上报kbroker_super模块并处理；
    kbroker_server模块根据其所在服务器CPU、内存、磁盘、网络的总体使用情况，对其上运行的app_allocator模块和app_service模块进行过载设置、迁移、关闭的管理操作。
  14. 根据权利要求1所述的一种kbroker分布式操作系统，其特征在于，app_allocator模块还用于创建管理应用程序的app_object对象：
    计算型应用程序的app_object对象的模式包括永久存在模式和临时存在模式，存储型应用程序的app_object对象的模式为永久存在模式；
    app_allocator模块通过app_service模块执行创建启动app_object对象；
    app_allocator模块保存app_object对象的路由信息，路由信息为app_object对象运行在哪个app_service模块上，并对外提供请求路由信息的接口；
    app_allocator模块对于永久存在模式的app_object对象，提供相应的接口函数对其进行创建和删除；
    永久存在模式的app_object对象当被访问时启动运行，长时间没有访问请求的情况下app_object对象将被关闭并退出运行；
    对于管理永久存在模式app_object对象的app_allocator模块，包括一个存储型资源用于对已存在的app_object对象进行保存；
    对于临时存在模式的app_object对象，由app_allocator模块根据需要自行分配；
    app_allocator模块通过添加专属功能满足应用程序对app_object模块的特定要求。
  15. 根据权利要求14所述的一种kbroker分布式操作系统，其特征在于，存储型应用程序的app_allocator模块维护app_service模块的主从灾备：
    存储型应用程序的app_allocator模块把若干个相应的app_service模块编成一组，并对每个分组设置分组编号virtual_id，维护每个分组内app_service模块的主从灾备；
    存储型应用程序的app_service模块都设有与之对应的存储型资源，存储型应用程序的app_allocator模块保存分组的分组编号virtual_id与该分组内app_service模块用到的存储型资源的资源编号resource_id的关联关系；
    已存在的app_object对象与分组一一对应绑定，存储型应用程序的app_allocator模块保存该对应关系；新的app_object对象在申请创建时选择一个分组进行绑定，当没有可用的分组时先创建新的分组和分组内的app_service模块，然后再选择一个分组进行绑定；
    存储型应用程序的app_allocator模块在处理对app_object对象的路由信息请求时，查找到该app_object对象对应分组的分组编号virtual_id后，找到该分组编号virtual_id对应分组内的主app_service模块的进程编号program_id返回给请求方。
  16. 根据权利要求1所述的一种kbroker分布式操作系统，其特征在于，app_service模块 运行app_object对象的具体逻辑：
    app_object对象由应用程序的应用程序编号app_id以及app_object对象在应用程序上的对象编号object_id来唯一标识，业务层通过应用程序的应用程序编号app_id以及app_object对象在应用程序上的对象编号object_id对app_object对象进行访问；
    业务层通过重写app_service模块提供的app_object_i虚基类来实现业务逻辑；
    单个app_object对象的逻辑处理只能运行在单个app_service模块上，单个app_service模块上同时运行多个app_object对象；
    kbroker分布式操作系统提供通过对象编号object_id创建app_object对象的虚接口，app_service模块通过业务层重写的所述虚接口创建app_object对象，通过app_object对象的初始化接口初始化app_object对象；
    同一个app_object对象的业务逻辑在app_service模块上的同一个线程上排队进行处理，app_object对象的每一个任务事件都封装成协程进行执行，kbroker分布式操作系统提供对阻塞型接口的协程化支持；
    app_service模块对app_object对象路由信息进行缓存，通过app_object对象对应的app_allocator模块获取路由信息来更新缓存，并在其他app_object对象访问时更新访问请求中来源app_object对象路由信息的缓存；
    业务层对外访问基于app_object对象，系统提供访问外部app_object对象的接口，该接口由app_service模块实现；app_service模块根据缓存或从对应app_allocator模块请求获得的路由信息找到app_object对象运行所在app_service模块的进程编号program_id，发送目标为该进程编号program_id的消息使得该消息发送到app_object对象运行所在的app_service模块；
    app_service模块对访问外部app_object对象的远程函数调用提供协程化支持；
    app_service模块收到app_object对象待处理消息后，检查目标app_object对象的运行状况并作处理；
    app_service模块在监控到其过载的情况下通知app_allocator模块不再分配新的app_object对象到其上运行；
    app_service模块提供用于业务层添加异步事件和延时事件的接口，协程化的锁和协程化的信号；
    app_service模块支持将正在运行的app_object对象迁移到与其同一应用程序的其他app_service模块上；
    存储型应用程序的app_service模块提供针对app_object对象的主从灾备框架，与业务层 重写的app_object_i虚基类中主从同步相关的接口配合实现主从灾备功能；
    存储型应用程序的app_service模块提供用于主从app_object对象间传递数据的接口。
  17. 根据权利要求16所述的一种kbroker分布式操作系统，其特征在于，业务层通过重写app_object_i虚基类来实现业务逻辑：
    计算型应用程序的app_object_i虚基类和存储型应用程序的app_object_i虚基类都包括function_api接口、notify_api接口、初始化接口、删除接口、导出数据接口、导入数据接口；function_api接口用于处理需要返回值的访问请求，notify_api接口用于处理不需要返回值的访问请求；
    业务层模块通过重写function_api接口、notify_api接口实现业务逻辑处理；业务层模块重写初始化接口用于app_service模块创建app_object对象后执行初始化；业务层模块重写删除接口用于app_service模块在关闭app_object对象时释放业务层创建的数据；导出数据接口和导入数据接口用于app_object对象的迁移，默认情况下为不支持app_object对象迁移，如果业务层重写这两个接口则该应用程序支持app_object对象的迁移，当业务负载高时将app_object对象迁移到负载低的app_service模块上；
    存储型应用程序的app_service模块实现主从灾备，其app_service模块实现针对其存储型资源的数据的整体导出导入接口和主从切换接口，另外其app_object_i虚基类还包括主从同步接口，主从同步接口配合app_service模块的主从灾备框架实现业务层逻辑数据的主从灾备。
  18. 根据权利要求1所述的一种kbroker分布式操作系统，其特征在于，所述网关模块包括session_allocator模块、kbroker_gateway模块；
    所述session_allocator模块基于app_allocator模块的基础功能进行扩展开发，保存所述kbroker_gateway模块的socket监听信息，实现多个所述kbroker_gateway模块的负载均衡；
    所述kbroker_gateway模块基于app_service模块的基础功能进行扩展开发，支持各种基于网络的外部请求访问方式，在其上运行的app_object对象为被动创建且包括对象编号object_id的外部请求连接；
    所述kbroker_gateway模块用于接收外部请求，并从请求中解析出请求对应目标app_object对象的应用程序编号app_id和对象编号object_id；
    所述kbroker_gateway模块支持对外部请求设定默认消息处理策略和基于请求对应目标app_object对象的应用程序编号app_id的特殊处理策略，通过外部请求中解析出来的目标app_object对象的应用程序编号app_id选择确定处理策略；如果有特殊处理策略则执行特别 处理策略，否则执行默认处理策略；
    所述kbroker_gateway模块还包括提供给系统内其他应用程序访问的notify_api接口、function_api接口；所述notify_api接口、function_api接口组成了系统内消息处理策略，所述系统内消息处理策略为默认消息处理策略或基于消息来源app_object对象的应用程序编号app_id的特殊消息处理策略，支持设定针对某个应用程序编号app_id的特殊消息处理策略。
  19. 根据权利要求1所述的一种kbroker分布式操作系统，其特征在于，kbroker分布式操作系统包括一单例应用程序：
    所述单例应用程序管理kbroker分布式操作系统内的单例型业务逻辑，所述单例型业务逻辑为不能分配到多个地方而只在一个程序进程中执行的业务逻辑；
    所述单例应用程序为kbroker分布式系统中一个特殊的应用程序；所述单例应用程序的应用程序编号为系统特殊分配的单例应用程序编号singleton_app_id；所述单例应用程序的app_service模块为各种功能不一的单例型业务逻辑的程序进程；
    所述单例应用程序的app_allocator模块为每个app_service模块分配一个单例对象编号singleton_object_id，用于app_service模块相对于其他应用程序展现为单例应用程序的一个对象，业务层通过单例应用程序编号singleton_app_id和app_service模块的单例对象编号singleton_object_id访问单例应用程序的app_service模块；
    所述单例应用程序的app_service模块支持主从灾备，app_allocator模块管理维护app_service模块的主从灾备；
    所述单例应用程序的app_service模块包括一个唯一的业务层app_object对象，所有业务层的请求都由这个app_object对象处理；
    所述单例应用程序的app_allocator模块由kbroker分布式操作系统提供，kbroker_super模块和kbroker_server模块对启动所述单例应用程序提供支持。
  20. 根据权利要求1所述的一种kbroker分布式操作系统，其特征在于，kbroker_super模块支持主从灾备，当从kbroker_super模块不可用时，主kbroker_super模块启动新的从kbroker_super模块实现替代；当主kbroker_super模块不可用时，所有从kbroker_super模块选举确定其中一个从kbroker_super模块切换成主kbroker_super模块，然后再启动新的从kbroker_super模块实现替代。
  21. 根据权利要求1所述的一种kbroker分布式操作系统，其特征在于，还包括管理后台， 用于操作kbroker分布式操作系统：
    管理后台提供访问接口给kbroker分布式操作系统，kbroker分布式操作系统通过配置或启动参数在启动时获得该访问接口；kbroker分布式操作系统通过该接口上报状态信息，状态信息包括管理网关的连接信息、服务器运行的监控信息、各个应用程序运行的状况、针对异常情况的报警、针对特定状况的提前预警；
    管理后台通过管理网关获取kbroker分布式操作系统的运行信息和发送操作指令管理kbroker分布式操作系统；
    管理后台在收到kbroker分布式操作系统上报的报警或预警信息时，报警通知运维和开发人员进行处理。
  22. 根据权利要求1所述的一种kbroker分布式操作系统，其特征在于，在支持更大规模的服务器集群时，kbroker分布式操作系统抽出一个kbroker_messager模块进行消息转发：
    kbroker_messager模块基于进程编号program_id进行消息转发；
    系统中的每个kbroker_server模块都与一个kbroker_messager模块通信连接，kbroker_messager模块保存与其相连的kbroker_server模块的进程编号program_id以及在其服务器上运行的其他程序进程的进程编号program_id，将这些进程编号program_id标记为归自己管理，并将这些进程编号program_id和其与该kbroker_server模块的通信连接进行绑定；
    多个kbroker_messager模块之间通信连接，kbroker_messager模块保存其他与之相连的kbroker_messager模块上管理的进程编号program_id，并将进程编号program_id和所属kbroker_messager模块与其之间的通信连接进行绑定；
    kbroker_super模块管理kbroker_messager模块，分配每一个kbroker_server模块连接到一个kbroker_messager模块；
    当使用kbroker_messager模块进行消息转发时，kbroker_server模块之间不进行通信连接，kbroker_server模块在收到其管理的程序进程发送的消息后转发给kbroker_messager模块。
  23. 一种存储介质，其上存储有计算机程序，其特征在于，该程序被处理器执行时以实现权利要求1-22所述的一种kbroker分布式操作系统。
  24. 一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其特征在于，所述处理器执行所述程序时以实现权利要求1-22所述的一种kbroker分布式操作系统。
PCT/CN2020/112816 2019-09-06 2020-09-01 一种kbroker分布式操作系统、存储介质和电子设备 WO2021043124A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910843920.2A CN110543315B (zh) 2019-09-06 2019-09-06 一种kbroker分布式操作系统、存储介质和电子设备
CN201910843920.2 2019-09-06

Publications (1)

Publication Number Publication Date
WO2021043124A1 true WO2021043124A1 (zh) 2021-03-11

Family

ID=68712876

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/112816 WO2021043124A1 (zh) 2019-09-06 2020-09-01 一种kbroker分布式操作系统、存储介质和电子设备

Country Status (2)

Country Link
CN (1) CN110543315B (zh)
WO (1) WO2021043124A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110543315B (zh) * 2019-09-06 2021-08-31 程延辉 一种kbroker分布式操作系统、存储介质和电子设备
CN111104162B (zh) * 2019-12-18 2023-03-24 程延辉 一种新旧代码共同运行的kbroker分布式操作系统
CN115202882B (zh) * 2022-07-26 2023-11-03 上海中汇亿达金融信息技术有限公司 分布式应用架构和该架构的执行方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170344393A1 (en) * 2016-05-31 2017-11-30 Huawei Technologies Co., Ltd. Virtual machine resource utilization in a data center
CN107888657A (zh) * 2017-10-11 2018-04-06 上海交通大学 低延迟分布式存储系统
CN108696578A (zh) * 2018-04-26 2018-10-23 昆明理工大学 基于ZeroMQ的多机分布式计算的通信框架设计方法
CN110543315A (zh) * 2019-09-06 2019-12-06 程延辉 一种kbroker分布式操作系统、存储介质和电子设备
CN111104162A (zh) * 2019-12-18 2020-05-05 程延辉 一种新旧代码共同运行的kbroker分布式操作系统

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101996094A (zh) * 2009-08-12 2011-03-30 中兴通讯股份有限公司 一种分布式资源管理方法及系统
CN103118124B (zh) * 2013-02-22 2015-08-05 桂林电子科技大学 一种基于分层多代理的云计算负载均衡方法
US9921910B2 (en) * 2015-02-19 2018-03-20 Netapp, Inc. Virtual chunk service based data recovery in a distributed data storage system
CN107277144B (zh) * 2017-06-22 2021-02-09 浙江力石科技股份有限公司 一种分布式高并发云存储数据库系统及其负荷均衡方法
CN108776640A (zh) * 2018-05-07 2018-11-09 深圳壹账通智能科技有限公司 分布式测试方法、装置、计算机设备和存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170344393A1 (en) * 2016-05-31 2017-11-30 Huawei Technologies Co., Ltd. Virtual machine resource utilization in a data center
CN107888657A (zh) * 2017-10-11 2018-04-06 上海交通大学 低延迟分布式存储系统
CN108696578A (zh) * 2018-04-26 2018-10-23 昆明理工大学 基于ZeroMQ的多机分布式计算的通信框架设计方法
CN110543315A (zh) * 2019-09-06 2019-12-06 程延辉 一种kbroker分布式操作系统、存储介质和电子设备
CN111104162A (zh) * 2019-12-18 2020-05-05 程延辉 一种新旧代码共同运行的kbroker分布式操作系统

Also Published As

Publication number Publication date
CN110543315A (zh) 2019-12-06
CN110543315B (zh) 2021-08-31

Similar Documents

Publication Publication Date Title
US11687555B2 (en) Conditional master election in distributed databases
WO2021043124A1 (zh) 一种kbroker分布式操作系统、存储介质和电子设备
US9442749B2 (en) Transaction-based service control system and control method
US8555242B2 (en) Decentralized system services
JP4637842B2 (ja) クラスタ化されたコンピューティングシステムにおける高速なアプリケーション通知
US9787561B2 (en) System and method for supporting a selection service in a server environment
US8930527B2 (en) High availability enabler
US9749445B2 (en) System and method for updating service information for across-domain messaging in a transactional middleware machine environment
US20120239730A1 (en) System including a middleware machine environment
US20010042139A1 (en) Replicated resource management system for managing resources in a distributed application and maintaining a relativistic view of state
KR20170097070A (ko) 통신 이벤트 달성
JP2015537307A (ja) コンポーネント指向ハイブリッドクラウドオペレーティングシステムのアーキテクチャ及びその通信方法
CN111143054A (zh) 一种异构国产cpu资源融合管理方法
CN112698838B (zh) 多云容器部署系统及其容器部署方法
US10721335B2 (en) Remote procedure call using quorum state store
CN111158949A (zh) 容灾架构的配置方法、切换方法及装置、设备和存储介质
CN114448983A (zh) 基于ZooKeeper的分布式数据交换方法
Mohamed et al. MidCloud: an agent‐based middleware for effective utilization of replicated Cloud services
CN114615268B (zh) 基于Kubernetes集群的服务网络、监控节点、容器节点及设备
KR102119456B1 (ko) 분산 클라우드 환경에서의 분산 브로커 코디네이터 시스템 및 방법
CN116723077A (zh) 一种分布式it自动化运维系统
CN114598700A (zh) 通信方法及通信系统
US20240143448A1 (en) Distributed cloud system, data processing method of distributed cloud system, and storage medium
CN113973049B (zh) 一种fpga集群管理与部署比特流的方法
CN114124680B (zh) 一种文件访问控制告警日志管理方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20861085

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20861085

Country of ref document: EP

Kind code of ref document: A1