CN113486099A - Memory computing service framework and implementation system - Google Patents
Memory computing service framework and implementation system Download PDFInfo
- Publication number
- CN113486099A CN113486099A CN202110702160.0A CN202110702160A CN113486099A CN 113486099 A CN113486099 A CN 113486099A CN 202110702160 A CN202110702160 A CN 202110702160A CN 113486099 A CN113486099 A CN 113486099A
- Authority
- CN
- China
- Prior art keywords
- service
- query
- framework
- service framework
- message
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 claims abstract description 37
- 238000003860 storage Methods 0.000 claims abstract description 37
- 238000004891 communication Methods 0.000 claims abstract description 24
- 238000004364 calculation method Methods 0.000 claims abstract description 19
- 238000000034 method Methods 0.000 claims abstract description 16
- 238000013500 data storage Methods 0.000 claims abstract description 14
- 230000002688 persistence Effects 0.000 claims abstract description 12
- 230000008859 change Effects 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 11
- 230000005540 biological transmission Effects 0.000 claims description 8
- 238000013499 data model Methods 0.000 claims description 8
- 239000000284 extract Substances 0.000 claims description 6
- 230000000977 initiatory effect Effects 0.000 claims 1
- 238000000926 separation method Methods 0.000 abstract description 18
- 230000003993 interaction Effects 0.000 abstract description 6
- 238000012544 monitoring process Methods 0.000 abstract description 4
- 238000005516 engineering process Methods 0.000 abstract description 2
- 238000011161 development Methods 0.000 description 19
- 238000007726 management method Methods 0.000 description 14
- 239000008186 active pharmaceutical agent Substances 0.000 description 13
- 230000007246 mechanism Effects 0.000 description 10
- 238000013461 design Methods 0.000 description 8
- 230000014509 gene expression Effects 0.000 description 8
- 230000002159 abnormal effect Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000005538 encapsulation Methods 0.000 description 4
- 230000010354 integration Effects 0.000 description 4
- 238000007405 data analysis Methods 0.000 description 3
- 230000001419 dependent effect Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000003672 processing method Methods 0.000 description 3
- 230000005856 abnormality Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000007474 system interaction Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 238000013024 troubleshooting Methods 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/547—Messaging middleware
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a memory computing service framework and an implementation system. The service framework takes Nats as a message bus for communication between services; the service triggers business logic processing by subscribing the specified topic, and releases the memory state of the service framework to the specified topic in a message form after the processing is finished; the service framework uses a Hazelcast cache service state to provide uniform persistence and query service for memory state persistence and query; SQL is used for uniformly expressing query and subscription conditions, and a Q & S subscription mode is creatively introduced to ensure that message consumption is not lost and repeated. The data storage and query are logically decoupled from the service and read-write separation is realized through a predefined storage model and a unified storage query service, the problems of data interaction usability and reliability are solved by introducing technologies such as SQL, Q & S and the like, and the method is mainly applied to scenes such as transaction, wind control, bin space, real-time monitoring, real-time index calculation and the like and can be popularized as a standard framework of memory computing service.
Description
Technical Field
The invention belongs to the field of memory computing, and relates to a memory computing service framework and an implementation system.
Background
Under the background of the large tide of digitization, with the increase of data production speed and data volume and the improvement of the requirement on the real-time performance of data processing, the traditional data processing mode of 'landing first and then processing' cannot be accepted, data must be analyzed and processed in a very short time when being generated, then the output result is used as the input of other systems, a database is not positioned in the center of real-time data processing any more, but is retreated to the back of the screen to be used as a carrier for analyzing and operating the data after landing. However, in a system with a requirement on real-time performance, shadows of a database are difficult to see on a key path, most of the systems are based on a memory computing architecture, but the memory computing is not universal, and a plurality of pain points still exist to wait for solving.
Through search and discovery, chinese invention of CN106021484A discloses a customizable multimode big data processing system based on memory computing in 2016, 10, 12, which includes: the system comprises a data storage layer module, a memory-based data sharing and management layer module, a memory-computing-based general execution framework layer module and an access interface layer module; the system adopts a distributed memory abstraction mechanism, a position perception scheduling mechanism and a distributed mixed column type storage mechanism of data. The invention constructs mass data storage, provides a memory data management and sharing framework facing cluster concurrent computation, provides a high-efficiency and large-data multi-mode general processing framework customized according to needs, supports batch processing and real-time data stream computation, and provides support for flexible analysis and deep utilization of data. However, due to the lack of a message bus, the upstream message needs to rely on other services both logically and physically, so that logical decoupling and physical decoupling cannot be realized, and the execution time of the total task is relatively long; and no unified specification and practice is formed in the aspects of interface definition, calling mode, storage model and the like. Therefore, on the premise that the real-time requirement of data processing is greatly improved, a uniform service framework is required to provide basic capabilities of service management, message routing, subscription and publication, data storage and query and the like.
The invention of china with publication number CN105426504A discloses a distributed data analysis processing method based on memory calculation in 2016, 3, 23, which includes the following steps: 1) providing an SQL-like analyzer, analyzing the input query analysis text into a corresponding logic plan, and performing primary optimization; 2) providing a task converter, converting the logic plan generated by the SQL-like parser into a calculation expression which can be identified by a big data memory calculation model and comprises a plurality of self-defined conversion classes; 3) and providing a query optimizer, converting the expressions which can be identified by the transmitted memory calculation model into logic plans, optimizing the logic plans, and converting the logic plans into physical execution plans. Aiming at the vacancy of the traditional big data processing in data query analysis processing, the invention provides a distributed data analysis processing method based on memory computing, which inherits the advantages of a memory computing model in data processing and enables the programming language of data query analysis to be simpler. However, data is still dispersed in each service system, query requests of external systems must be processed by the service systems, read-write separation cannot be achieved, processing of service requests with higher priority and overall stability of service are affected, and a data analysis processing method needs to be improved.
At present, systems such as self-developed trading, wind control, bin position, real-time monitoring, real-time index calculation and the like of each team related to department trading business are all built based on the idea of memory calculation, and the performance of the systems is improved by orders of magnitude compared with a database system, but the systems also have the following problems:
(1) services are in war, and unified specifications and practices are not formed in the aspects of interface definition, calling mode, storage model and the like;
(2) data are dispersed in each service system, query requests of external systems must be processed by the service systems, read-write separation cannot be achieved, large-data-volume query may cause long-time GC pause, even the service is unavailable wholly, and processing of service requests with higher priority and stability of the service are affected;
(3) data interaction cost between services is high, the subscribing and querying API must be defined in advance, and a new query request can be satisfied only by adding the API, so that the problem of insufficient flexibility exists;
(4) the reliability mechanism during message consumption is imperfect, most services ensure the final consistency of the state through subscription, polling and duplicate removal, and the network bandwidth consumption and the load of a data producer are increased;
(5) in the platform construction process, the existing self-research system is docked through RPC calling, secondary development is carried out on an order management system based on a point-to-point private communication protocol, communication with a third-party system of a supplier is achieved, system interaction is complicated, and different calling modes, middleware and data models are involved.
Because of the above problems, a unified service framework is needed to provide basic capabilities of service management, message routing, subscription and publication, data storage and query, and improve the performance of memory computing.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a memory computing service framework, which not only solves the problem that interface definition, calling modes and storage models cannot form unified specifications, but also solves the problem that data in each service system cannot be read and written separately.
In one aspect, the invention provides a memory computing service framework, which takes Nats as a message bus for communication between services; the service triggers business logic processing by subscribing the specified topic, and releases the memory state of the service framework to the specified topic in a message form after the processing is finished; the service framework caches the service state using Hazelcast.
In the technical scheme, the memory computing service framework uses the Nats as a message bus for communication between services, so that the overall throughput capacity is improved, and the message transmission rate is ensured. Nats is an open-source high-performance message middleware, can achieve millions of throughputs per second, and meanwhile guarantees end-to-end message transmission delay within milliseconds. The problem of the brooker (message broker) based service communication is that a large communication delay is introduced, the delay of the Nats is far superior to other similar brookers and is only slightly slower than the point-to-point communication, and therefore the extremely high performance of the Nats is one of the important reasons for selecting the brooker (message broker) based service communication as a message bus. On the other hand, nat provides a filter expression, and provides a foundation for message routing and load balancing capabilities of the service framework. The fine-grained message iilter expression (in the form of a.b.. C) provided by Nats is also a characteristic that Nats are different from all similar brokers, and in the service framework, the capabilities of message routing, load balancing and the like are all constructed based on the characteristic. Meanwhile, all publishers and subscribers communicate through Nats, services are not directly connected with each other, and good support is provided for system expansibility. Meanwhile, topic is introduced on the basis of open source Nats, service discovery and communication mechanisms based on topic are realized, and the core management capability of the service framework is realized through topic management. All services trigger business logic processing by subscribing the specified topic, and then publish the memory state of the services to the specified topic in a message form for consumption by other services after the processing is finished. The service framework uses Hazelcast to cache the service state, and provides a foundation for a read-write separation architecture. Hazelcast is an open-source distributed cache and has index creation and maintenance and SQL-like query capacity, and the two characteristics form the basis of a read-write separation architecture in the service framework. Hazelcast is used as a global cache, the state and the change history of all services in a recent period are stored, the service state is written into the Hazelcast through persistent service and index is automatically maintained, the query capability is provided in an SQL mode through query service, and the query function is not required to be provided by business service, so that the burden of the business service is greatly reduced.
Preferably, in the service framework, the service uses topic to perform pub-sub mode communication based on Nats, and each topic includes the following attributes: name, type, coding mode of transmission information on topic, creator or owner of topic, template; when a data producer issues a message to a specified topic, the memory computing service framework SDK extracts a corresponding field value in the message according to the template attribute of the topic to form a subject, and Nats performs message routing according to the subject and subscription conditions. In the service framework, services use topic to carry out pub-sub mode communication based on Nats. topic is one of the most central concepts in the Nats framework, and each topic contains the following attributes: name, type, encoding (encoding mode of message transmitted on topic), owner (creator or owner of topic), template; wherein, type supports business and system, encoding can be json or pb, template is used as message route and metadata of filter; and template is the most critical attribute, and for the definition of one template: { trainaccount }, securityId }, exchange } { serviceName }, when a data producer issues a message to a specified topic, the memory computing service framework SDK extracts a corresponding field value in the message to form a subject according to a template attribute of the topic, and Nats routes the message according to the subject and subscription conditions. The definition of topic in the service framework is combined with the characteristic that Nats distributes messages according to subjects, so that very flexible service design can be formed. The subscription conditions are formed by combining according to any dimensionality, so that the flexibility of the service framework can be greatly improved, and meanwhile, the service layer can also carry out load balance design based on the characteristics of the service design.
Further, based on the service development of the service framework, the input and the output of the service are firstly combed, and then the created topic, the subscribed topic and the message format transmitted on the created and subscribed topic are defined. By combing the input and output of the service, defining which topics are created, which topics are subscribed and the message format transmitted on the topics, the purpose of restricting the service integration specification of the service framework is achieved, and on the premise of complying with the same specification, the framework layer can provide more standardized capacity and assist in the development of services. Therefore, the service framework forms a uniform specification in the aspects of interface definition, calling mode, storage model and the like, the performance of data processing is improved, and the service forms the same specification.
Preferably, the service framework comprises a management center node Monitor for managing the state of the service in the system and initializing the registration and login of the service; the service is started through login, registration, parameter synchronization and dependency check; the Monitor broadcasts a change in status for any one service. By setting the management center node Monitor, the management of the states of all the services in the service framework is realized, and the management of initialization operations such as service registration and login can also be realized. The Monitor broadcasts the state change of any one service, other services can subscribe to the change and make corresponding decisions, and if the state of the dependent service is abnormal, the service application can sense and perform corresponding processing. The manager can know the running state of the service in time, can perform corresponding processing on the service in different states, and improves the stability of the whole running of the service framework.
Furthermore, the service is started to be the INIT state, and the initialization is completed, the service is started successfully, and the state is updated to be the ACTIVE state through the processes of a) login, b) registration, c) synchronization parameters, d) dependency checking and the like. The service and the Monitor maintain the self state through the heartbeat, if the Monitor cannot receive the service heartbeat for a long time, the service state is judged to be abnormal, and the service state is updated to INACTIVE. After the service in the ACTIVE state is stopped, the system finally enters a SHUTDOWN state; and after the service in the INACTIVE state exits, the system finally enters a SHUTDOWN state. Different states and corresponding judgment conditions are set, so that the accuracy of the manager in monitoring the service state is ensured, and the problems of erroneous judgment or missing detection and the like are avoided. The manager can know the running state of the service in time, can carry out corresponding processing on the service in different states, and further improves the stability of the whole running of the service framework.
Preferably, the dependency check at service start-up is checked by whether the service status satisfies the dependency, the method is as follows: a) if the application is configured with dependency during starting, if the state of the depended party is not ACTIVE, the starting of the application is waited until the depended party returns to normal; b) if the application is configured with runtime dependence, the application receives the notification and processes when the state of the depended party changes. The constraint mode can help a manager to discover the abnormality between the services which are mutually dependent in time, grasp the running state of the services in time, perform targeted processing on the abnormal services, and further ensure the stability of the overall running of the service framework.
Preferably, the general data storage and query service is set based on Hazelcast and a database, an application service developed based on the service framework designates data needing to be persisted through annotation during data model definition, and a field of query index is established, the storage and query service provided by the memory computing service framework automatically persists the data and provides query, and the operation burden of the service is reduced. In the prior art, most queries need to be processed by a data producer due to the lack of a unified storage model and query service, in the service framework, a general data storage and query service is designed based on Hazelcast and a database, an application service developed based on the service framework only needs to specify which data need to be persisted through annotation in the definition of a data model and establish query indexes in which fields, and the storage and query service provided by the service framework can automatically persist the data and provide queries. Data storage and querying in the service framework becomes a universal capability without the need for re-attention of each service. The cost of data interaction between services is greatly reduced, and the flexibility of the service framework is improved.
Preferably, in the service framework, the Cache writer and the DB writer service subscribe to messages that need to be persisted, and simultaneously and independently write to the Cache and the database, respectively; the query request is processed by the query service of the framework layer, the data with high real-time requirement is queried from the cache, and meanwhile, the query from the database is supported. Under the architecture mode, read-write separation is realized, a large number of query requests do not need to be processed by the service, and the burden of the service is greatly reduced.
Preferably, in the service framework, one object has two storage models, namely Key-Value and relation type, and provides a uniform SQL query interface for the outside. SQL is a structured query language, usually used in combination with a relational storage model, and in the service framework, an object has two storage models, i.e., Key-value (Hazelcast) and relational (mysql), and the interface of the object is simply encapsulated by one layer by using the characteristic that Hazelcast supports SQL-like query, and a uniform SQL query interface is provided externally, so that data stored in the service framework can be queried through one SQL regardless of a Cache (Hazelcast Cache) or a Database (Database), thereby greatly reducing the use threshold and reducing the service development cost.
Preferably, the service framework unifies the subscription and query interfaces by encapsulating the Nats API and expressing the subscription conditions with SQL. Besides query, the application layer also subscribes real-time data in the service framework, the native API provided by the Nats needs a caller to construct a subscription expression like A.B. C, the usability is low, and the service framework creatively provides and realizes subscription conditions expressed by SQL through encapsulation of the Nats API, so that the subscription and query interfaces are unified, and the business development threshold is further reduced. Different scenes have different semantics for the same SQL statement. The service framework provides uniform SQL interfaces for different data acquisition scenes, is one of the most valuable innovation points, reduces the service development cost, enables a third-party tool to be seamlessly accessed, and simultaneously realizes load balancing through different subscription SQL for different instances of the same service.
Preferably, the Service framework adopts a Query Service atomic subscription mode. In practical applications, there is a special subscription scenario, and a user needs to receive not only a change push of an object state after a subscription takes effect, but also a latest state of the object before the subscription. For this special subscription scenario, the Service framework adopts a solution of Query Service atomic subscription mode. Whereas heretofore, a more common solution has been a Query-only (Query) schema or a subscription-only (Subscribe) schema: in contrast, the Query Service atomic subscription mode not only saves network bandwidth, but also reduces the pressure brought by the market Service, and can more efficiently process the subscription requirements of the user.
In another aspect, the present invention provides a system for implementing a memory computing service framework, including:
an access layer: supporting the request of the client, and interacting with each service through a message bus;
an application layer: the system comprises platform services and business services, wherein the platform services support a business system, and the business services are uniformly accessed to a message bus through a memory computing service framework (SDK);
a storage layer: the system comprises a cache and a database, wherein a storage layer is used for data persistence;
the platform service is driven by a message event, a service application generates a message through memory calculation, and the platform service is responsible for storing and inquiring the message.
The access layer is mainly gateway, supports the request of the client and interacts with each service through Nats. The platform service in the application layer supports all business systems; and the business service is uniformly accessed to the Nats through a memory computing service framework (SDK). The storage layer is focused on data persistence and is transparent to a service user. The whole platform is driven by message events, the service application generates messages through memory calculation, and the platform service is responsible for storing and inquiring the messages. Through the setting of the system, the read-write separation is realized, the business service does not need to process a large number of inquiry requests any more, and the burden of the service is greatly reduced. The system implements the features and functions of a memory computing service framework: by using Nats as a message bus for communication between services and using Hazelcast cache service state, unified persistence and query service is provided for memory state persistence and query, and a Q & S subscription mode is introduced to ensure that message consumption is not lost and repeated.
Compared with the prior art, the invention has the beneficial effects that:
(1) the memory computing service framework of the invention covers the aspects of message driving, memory computing, read-write separation, load balancing and the like, and completely constructs a whole set of service system supporting the memory computing. The memory computing service framework of the invention takes Nats as a message bus for communication between services, thereby improving the throughput capacity of the service framework and ensuring the message transmission rate. All publishers and subscribers communicate through Nats, topic is creatively introduced on the basis of open-source Nats, and service discovery and communication mechanisms based on topic and stateless service load balancing based on topic group characteristics are achieved. All services trigger service logic processing by subscribing the specified topic, and after the processing is finished, the memory state of the service framework is released to the specified topic in a message form for consumption of other services, the services are not directly connected with each other, and good support is provided for the expansibility of the system. The service framework uses Hazelcast to cache the service state, and provides a foundation for a subsequent read-write separation architecture.
(2) According to the memory computing service framework, all services use topic to carry out pub-sub mode communication based on Nats, and each topic comprises a name, a type, a coding mode of transmission information on the topic, a creator or owner of the topic, a template and other attributes; when a data producer issues a message to a specified topic, the memory computing service framework SDK extracts a corresponding field value in the message according to the template attribute of the topic to form a subject, and Nats performs message routing according to the subject and subscription conditions. the definition of topic combines the characteristic that Nats distributes messages according to subjects, and very flexible service design can be formed. The subscription conditions are formed by combining according to any dimensionality, so that the flexibility of the service framework can be greatly improved, and meanwhile, the service layer can also carry out load balance design based on the characteristics of the service design.
(3) The memory computing service framework constructs unified storage and query service on the basis of open-source Hazelcast, and uniformly expresses subscription and query by SQL, thereby greatly reducing the use threshold, improving the usability of the service framework, reducing the integration cost of other SQL tools and achieving the effect of read-write separation. And an application service developed based on the service framework specifies data needing to be persisted through annotation during data model definition, and establishes fields of a query index, and the storage and query service provided by the memory computing service framework automatically persists the data and provides query. The data storage and query in the memory computing service framework of the invention becomes a universal capability without paying attention to each service. Through the encapsulation of the Nats API, the SQL is creatively provided and used for expressing the subscription conditions, so that the subscription and query interfaces are unified, and the service development threshold is further reduced. The method and the system not only reduce the business development cost, but also enable a third-party tool to be seamlessly accessed, and simultaneously realize load balancing through different subscription SQL for different instances of the same service.
(4) The memory computing service framework creatively provides a Query & Subscribe (Q & S) atomic subscription mode, solves the problem that the subscription information of an application layer may lose data, and also solves the problem that the information is lost in the information subscription and Query intervals.
(5) The invention discloses a memory computing service framework implementation system, which comprises: the access layer is used for supporting the request of the client and interacting with each service through a message bus; the application layer comprises platform services and business services, the platform services support a business system, and the business services are uniformly accessed to a message bus through a memory computing service framework (SDK); the storage layer comprises a cache and a database, and the storage layer focuses on data persistence; the platform service is driven by a message event, the business application generates a message through memory calculation, and the platform service is responsible for storing and inquiring the message. The system enables the memory computing service framework to be realized, solves the common problems that the memory computing has no unified specification and practice, the read-write separation can not be realized, the data interaction cost between services is high, the reliable mechanism during message consumption is imperfect and the like, enables the service framework to have better expansion capability, and provides the SDK supporting the rapid development.
Drawings
FIG. 1 is a block diagram of a memory computing service framework implementation system according to an embodiment of the present invention;
fig. 2 is a block diagram of a scenario in which a memory computing service framework supports service publishing and subscribing according to an embodiment of the present invention;
FIG. 3 is a service state transition diagram of a memory computing service framework according to an embodiment of the present invention;
FIG. 4 is a tool interface diagram of a memory computing service framework according to an embodiment of the present invention;
fig. 5 is a block diagram of a Q & S subscription implementation mechanism of a memory computing service framework according to an embodiment of the present invention.
Detailed Description
The technical solutions of the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings, and it is to be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without any inventive step, are within the scope of the present invention.
Example one
Referring to fig. 1, the present embodiment provides a memory computing service framework, where the service framework uses Nats as a message bus for inter-service communication; the service triggers business logic processing by subscribing the specified topic, and after the processing is finished, the memory state of the service framework is released to the specified topic in a message form for consumption by other services; the service framework caches the service state using Hazelcast.
In this embodiment, a general data storage and query service is set based on Hazelcast and a database; and an application service developed based on the service framework specifies data needing to be persisted through annotation during data model definition, and establishes fields of a query index, and the storage and query service provided by the memory computing service framework automatically persists the data and provides query.
In the embodiment, the Cache writer and the DB writer service subscribe the message needing to be persisted and independently write the message into the Cache and the database at the same time; the query request is processed by the query service of the framework layer, the data with high real-time requirement is queried from the cache, and meanwhile, the query from the database is supported.
The memory computing service framework provided by this embodiment covers the aspects of message driving, memory computing, read-write separation, load balancing, and the like, and completely constructs a whole set of service system supporting memory computing. Nats is used as a message bus for communication between services, so that the overall throughput capacity is improved, and the message transmission rate is ensured. In addition, the fine-grained message filter expression (in the form of a.b.. C), message routing, load balancing and other capabilities provided by Nats are all constructed based on the characteristic. All publishers and subscribers communicate through Nats, all services trigger service logic processing by subscribing to the specified topic, and after the processing is completed, the memory state of the service framework is published to the specified topic in a message form for consumption of other services, the services are not directly connected with each other, and good support is provided for the expansibility of the system. On the basis of open source Nats, topic is introduced, a service discovery and communication mechanism based on topic is realized, the core management capability of the service framework is realized through topic management, and stateless service load balancing based on topic group characteristics is realized. The service framework uses Hazelcast to cache the service state, and provides a foundation for a read-write separation architecture.
In the absence of a unified storage model and query service, most queries need to be processed by a data producer, while in the service framework of the embodiment, a general data storage and query service is designed based on Hazelcast and a database, an application service developed based on the service framework only needs to specify which data need to be persisted through annotation in the definition of a data model and establish query indexes in which fields, and the storage and query service provided by the service framework can automatically persist the data and provide queries. Data storage and querying in the service framework becomes a universal capability without the need for re-attention of each service. Subscribing the message needing persistence by the Cache writer and the DB writer service, and simultaneously and independently writing the message into a Cache and a database respectively; the query request is processed by the query service of the framework layer, the data with high real-time requirement is queried from the cache, and meanwhile, the query from the database is supported. Under the architecture mode, read-write separation is realized, a large number of query requests do not need to be processed by the service, and the burden of the service is greatly reduced.
The embodiment provides a system for implementing a memory computing service framework, which includes:
an access layer: supporting the request of the client, and interacting with each service through a message bus;
an application layer: the system comprises platform services and business services, wherein the platform services support a business system, and the business services are uniformly accessed to a message bus through a memory computing service framework (SDK);
a storage layer: the system comprises a cache and a database, wherein a storage layer is used for data persistence;
the platform service is driven by a message event, a service application generates a message through memory calculation, and the platform service is responsible for storing and inquiring the message.
The access layer is mainly gateway, supports the request of the client and interacts with each service through Nats. The platform service in the application layer supports all business systems; and the business service is uniformly accessed to the Nats through a memory computing service framework (SDK). The storage layer is focused on data persistence and is transparent to a service user. The whole platform is driven by message events, the service application generates messages through memory calculation, and the platform service is responsible for storing and inquiring the messages. Through the setting of the system, the read-write separation is realized, the business service does not need to process a large number of inquiry requests any more, and the burden of the service is greatly reduced. The whole framework covers the aspects of message driving, memory calculation, read-write separation, load balancing and the like, and can completely construct a whole service system supporting the memory calculation.
Example two
Referring to fig. 2, in the memory computing service framework provided in this embodiment, a service uses topics to perform pub-sub mode communication based on Nats, and each topic includes the following attributes: name, type, coding mode of transmission information on topic, creator or owner of topic, template; when a data producer issues a message to a specified topic, the memory computing service framework SDK extracts a corresponding field value in the message according to the template attribute of the topic to form a subject, and Nats performs message routing according to the subject and subscription conditions.
In this embodiment, the service uses topic to perform pub-sub mode communication based on Nats, and topic management is a core management capability of the service framework. topic is one of the most central concepts in the Nats framework, and each topic contains the following attributes: name, type, encoding (encoding mode of message transmitted on topic), owner (creator or owner of topic), template; wherein, type supports business and system, encoding can be json or pb, and template is used as the metadata of message routing and filter. And template is the most critical attribute, and for the definition of one template:
{ trainaccount }, securityId }, exchange } { serviceName }, when a data producer issues a message to a specified topic, the memory computing service framework SDK extracts corresponding field values in the message according to a template attribute of the topic to form a subject (e.g., 001120.601688.sh. orderservice), and Nats routes the message according to the subject and subscription conditions. If there is an order request from the customer that the customer subscribes to all 001120 accounts ('001120'), then the message is forwarded to the customer for processing.
The definition of topic in the service framework is combined with the characteristic that Nats distributes messages according to subjects, so that very flexible service design can be formed. In the order processing scenario, we can combine dimensions such as trading account number (trading account) and market (exchange) into a subject. traderA issues an order to order, namely tradera.sh, and subscribers choose to subscribe to Shanghai (SH) only or traderA account only, and can receive the order and not other accounts or markets. As shown in fig. 2, these dimensions can be combined according to any dimension to form a subscription condition, which is very flexible, and the service layer can also perform load balancing design based on this characteristic.
Therefore, based on the service development of the service framework, the first step is to comb what the input and output of the service are, define which topics are created, which topics are subscribed, and the message format transmitted on the topics, so that the constraint forms the service integration specification of the service framework provided by the embodiment, and on the premise of complying with the same specification, the framework layer can provide more standardized capability-assisted service development.
EXAMPLE III
Referring to fig. 3, the memory computing service framework provided in this embodiment includes a management center node Monitor, where the node manages states of all services in the framework, and initialization operations such as service registration and login. The service start (INIT state) needs to go through the processes of a) login, b) registration, c) synchronization parameters, d) checking dependence and the like, and finally the start is successful (ACTIVE state). Referring to fig. 3, the service itself and the Monitor maintain their own states through heartbeat, and if the Monitor cannot receive the service heartbeat for a long time, it will determine that the service state is INACTIVE. After the service in the ACTIVE state is stopped, the system finally enters a SHUTDOWN state; and after the service in the INACTIVE state exits, the system finally enters a SHUTDOWN state. The Monitor broadcasts the state change of any one service, other services can subscribe to the change and make corresponding decisions, and if the state of the dependent service is abnormal, the service application can sense and perform corresponding processing.
In the memory computing service framework provided in this embodiment, the dependency check when the application is started is verified by whether the service state satisfies the dependency: a) if the application is configured with dependency in starting, if the state of the depended party is not ACTIVE, the starting of the application is waited until the depended party returns to normal; b) if the application is configured with runtime dependence, when the state of the depended party changes, the application receives notification and processes the state of the depended party in a targeted mode. The constraint mode can help an administrator to find abnormality among the interdependent services in time, grasp the running state of the services in time and carry out targeted processing on the abnormal services, so that the stability of the overall running of the service framework is improved.
Example four
Referring to fig. 4, in the memory computing service framework provided by this embodiment, an object has two storage models, i.e., Key-Value and relationship, and provides a uniform SQL query interface for the outside. SQL is a structured query language, usually used in combination with a relational storage model, and in the service framework, an object has two storage models, i.e., Key-value (Hazelcast) and relational (mysql), and the interface of the object is simply encapsulated by one layer by using the characteristic that Hazelcast supports SQL-like query, and a uniform SQL query interface is provided externally, so that data stored in the service framework can be queried through one SQL regardless of a Cache (Hazelcast Cache) or a Database (Database), thereby greatly reducing the use threshold and reducing the service development cost.
In the memory computing service framework provided by this embodiment, the service framework uses SQL to express subscription conditions by encapsulating the Nats API, and unifies the subscription and query interfaces. Besides query, the application layer also subscribes real-time data in the service framework, the native API provided by the Nats needs a caller to construct a subscription expression like A.B. C, the usability is low, and the service framework creatively provides and realizes subscription conditions expressed by SQL through encapsulation of the Nats API, so that the subscription and query interfaces are unified, and the business development threshold is further reduced. The problem of among the prior art data interaction cost height between service is solved. In the existing memory calculation, because the interfaces for subscription and query are not uniform, the APIs for subscription and query must be defined in advance, and a new query request must be satisfied by adding a new API, so that there is a pain point of insufficient flexibility.
In this embodiment, for the same SQL statement, different scenes have different semantics. The service framework provides uniform SQL interfaces for different data acquisition scenes, is one of the most valuable innovation points, reduces the service development cost, enables a third-party tool to be seamlessly accessed, and simultaneously realizes load balancing through different subscription SQL for different instances of the same service.
In the construction process of the existing platform, the existing self-research system is docked through RPC (remote procedure call), an order management system based on a point-to-point private communication protocol is designed for secondary development, the existing platform is communicated with a third-party system of a supplier, the interaction of the existing platform is complicated, and different calling modes, middleware and data models are involved. In the memory computing service framework of the embodiment, a layer of simple encapsulation is performed on the interface of the object by using the characteristic that the Hazelcast supports SQL-like query, so that a uniform SQL query interface can be provided for the outside. SQL is a structured query language that is typically used in conjunction with a relational storage model. In the memory computing service framework provided by this embodiment, each object has two storage models, namely, Key-value (hazelcast) and relational (mysql), so that data stored in the service framework provided by this embodiment can be queried through one SQL regardless of whether in a cache or a database, thereby greatly reducing the use threshold. The application layer can subscribe real-time data in the internal memory computing service framework provided by the embodiment except for query, the native API provided by the Nats needs to be called to construct a subscription expression like A.B. C, the usability is low, the internal memory computing service framework provided by the embodiment creatively provides and realizes that subscription conditions are expressed by SQL through packaging the Nats API, so that subscription and query interfaces are unified, the business development threshold is further reduced, the business development cost is reduced, a third-party tool can be seamlessly accessed, and different instances of the same service can realize load balance through different subscription SQL. For example, for the same SQL statement: select from the ordered hierarchy traffic of '001120' and exchange of 'SH', the following different scenarios have different semantics:
and (6) querying. Querying all exchange (SH) orders (orders) belonging to 001120 accounts;
and subscribing. Subscribing (SH) order (order) states of all 001120 accounts sent to an exchange;
q & S. And inquiring the latest state of the order meeting the conditions, and receiving a new state change push from the latest state.
The service framework of the embodiment provides uniform SQL interfaces for different data acquisition scenes, is one of the most valuable innovation points, reduces the service development cost, enables a third-party tool to be seamlessly accessed, and simultaneously realizes load balancing through different subscription SQL for different instances of the same service. Fig. 4 is a subscription query tool interface provided by the memory computing service framework of the embodiment. The SQL sentence ' select from order BO where trade date ' 20201230 ' is input in the message query box, so that corresponding transaction information can be searched and developed conveniently for problem tracking and troubleshooting; meanwhile, the inquiry, subscription and Q & S subscription of the message can be realized through the SQL statement, so that different scenes have uniform interfaces, and the use threshold is greatly reduced.
On the basis of open-source Hazelcast, unified storage and query services are constructed, subscription and query are uniformly expressed by SQL, so that the use threshold is greatly reduced, the usability of the service framework is improved, the integration cost of other SQL tools is reduced, and the effect of read-write separation is achieved. The problem that services are in conflict respectively and uniform specification and practice are not formed in the aspects of interface definition, calling mode, storage model and the like in the existing memory computing technology is solved; the problems that data are dispersed in each service system, read-write separation cannot be achieved, long-time GC pause can be caused by large-data-volume query, even the service is wholly unavailable, and processing of service requests with higher priority and overall service stability are affected are solved.
EXAMPLE five
Referring to fig. 5, the Service framework of the present embodiment adopts a Query Service atomic subscription mode (Q & S subscription). In practical applications, there is a special subscription scenario, and a user needs to receive not only a change push of an object state after a subscription takes effect, but also a latest state of the object before the subscription. Taking one scenario as an example: user a subscribes to the target market of "017856. IB" at 11 am, but since the transaction is not active, the latest status of this target is only 10: 58, the next change may be after 1 hour. At this time, if the user a passes the normal subscription, then since 10: if no push is available within 1 hour after 58, the targeted market can not be obtained. However, if Query Service is queried through the Service framework of this embodiment, the user a will also obtain the target value 10: 58, which is also the system behavior desired by the user. The Q & S subscription mode is a solution of the service framework in the present embodiment, and heretofore, a common solution is a Query-only (Query) mode or a subscription-only (Subscribe) mode: the user A can periodically inquire the targeted market data, and although the latest market can be obtained, the polling not only wastes network bandwidth, but also brings certain pressure to market service, and compared with the scheme of Q & S subscription, the scheme is more efficient.
The implementation concept is shown in fig. 5: at 10 times, t is 1, 2, 3 … 10, the targeted market information has 10 message flows in total. In the normal subscription mode, if the user inquires about the targeted market information at the time t-6, only the market data at the time t-1, 2, 3, 4, 5, 6 can be acquired; if the user subscribes to the target market information at the time t-6, 7, 8, 9 and 10 can be acquired, but the market information at the time t-1, 2, 3, 4 and 5 cannot be acquired. In the Q & S subscription mode, if the user subscribes to the target market information at time t 6, the user can acquire the target market information at time t 6, 7, 8, 9, 10 after subscription, and also acquire the target market data at time t 1, 2, 3, 4, 5, 6, which is the system behavior desired by the user. In a common subscription mode, a user can only periodically query the market quotation data, and although the latest market quotation can be obtained, the polling not only wastes network bandwidth, but also brings certain pressure to market quotation service. Therefore, the Query Service atomic subscription mode not only saves network bandwidth, but also reduces the pressure brought by market Service, and can more efficiently process the subscription requirements of users. The Q & S atomic subscription mode efficiently solves the data integrity problem in a series of scenes such as restart, disconnection reconnection and the like in a consumer disk.
In the existing memory calculation, the reliability mechanism during message consumption is imperfect, most services ensure the final consistency of the state through subscription, polling and duplicate removal, and the network bandwidth consumption and the load of a data producer are increased. The memory computing service framework of the embodiment creatively provides a Query & Subscribe atomic subscription mode (Q & S subscription), solves the pain point that the application layer subscribes messages and may lose data, and also solves the problem that messages are lost in message subscription and Query intervals.
The unified service framework provided by this embodiment provides basic capabilities of service management, message routing, subscription and publication, data storage and query, improves the performance of memory computation, and provides a standard development specification of memory computation service while liberating the productivity of services. The method is mainly applied to scenes such as trading, wind control, bin positions, real-time monitoring and real-time index calculation, and can be popularized as a standard framework of memory calculation service.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.
In the description herein, references to the description of the terms "one embodiment," "certain embodiments," "an illustrative embodiment," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the technical solutions of the present embodiments.
Claims (10)
1. A memory computing service framework is characterized in that the service framework takes Nats as a message bus for communication between services; the service triggers business logic processing by subscribing the specified topic, and releases the memory state of the service framework to the specified topic in a message form after the processing is finished; the service framework caches the service state using Hazelcast.
2. The memory computing service framework of claim 1, wherein services use topics for pub-sub mode based communication by Nats, each topic comprising the following attributes: name, type, coding mode of transmission information on topic, creator or owner of topic, template; when a data producer issues a message to a specified topic, the memory computing service framework SDK extracts field values in the message according to the template attribute of the topic to form a subject, and Nats performs message routing according to the subject and subscription conditions.
3. The memory computing service framework of claim 1, wherein the service framework comprises a management center node Monitor for managing the status of services in the system and initiating the registration and login of the services; the service is started through login, registration, parameter synchronization and dependency check; the Monitor broadcasts a change in status for any one service.
4. The in-memory computing service framework of claim 3, wherein the dependency check at service startup is checked by whether the service state satisfies the dependency by: a) if the application is configured with dependency during starting, if the state of the depended party is not ACTIVE, the starting of the application is waited until the depended party returns to normal; b) if the application is configured with runtime dependence, the application receives the notification and processes when the state of the depended party changes.
5. The memory computing service framework of claim 1, wherein a generic data storage and query service is provided based on Hazelcast and database; an application service developed based on the service framework specifies data needing to be persisted through annotation in data model definition, and establishes fields of a query index, and a storage and query service provided by the service framework automatically persists the data and provides a query.
6. The in-memory computing service framework of claim 5, wherein the service framework subscribes to messages needing persistence by the Cache writer and DB writer services and writes separately into the Cache and database at the same time; the query request is processed by the query service of the framework layer, the data with high real-time requirement is queried from the cache, and meanwhile, the query from the database is supported.
7. The in-memory computing service framework of claim 1, wherein an object in the service framework has two storage models, i.e. Key-Value and relational, and provides a uniform SQL query interface to outside.
8. The in-memory computing service framework of claim 1, wherein the service framework unifies the subscription and query interfaces by encapsulating the Nats API, expressing the subscription conditions in SQL.
9. The in-memory computing Service framework of claim 1, wherein the Service framework employs a Query Service atomic subscription model.
10. A memory computing service framework implementation system, comprising:
an access layer: supporting the request of the client, and interacting with each service through a message bus;
an application layer: the system comprises platform services and business services, wherein the platform services support a business system, and the business services are uniformly accessed to a message bus through a memory computing service framework (SDK);
a storage layer: the system comprises a cache and a database, wherein a storage layer is used for data persistence;
the platform service is driven by a message event, a service application generates a message through memory calculation, and the platform service is responsible for storing and inquiring the message.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110702160.0A CN113486099A (en) | 2021-06-23 | 2021-06-23 | Memory computing service framework and implementation system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110702160.0A CN113486099A (en) | 2021-06-23 | 2021-06-23 | Memory computing service framework and implementation system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113486099A true CN113486099A (en) | 2021-10-08 |
Family
ID=77936039
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110702160.0A Pending CN113486099A (en) | 2021-06-23 | 2021-06-23 | Memory computing service framework and implementation system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113486099A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115665191A (en) * | 2022-10-09 | 2023-01-31 | 浪潮云信息技术股份公司 | User information synchronization method and system based on cloud storage system |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107229639A (en) * | 2016-03-24 | 2017-10-03 | 上海宝信软件股份有限公司 | The storage system of distributing real-time data bank |
| CN109241054A (en) * | 2018-08-02 | 2019-01-18 | 成都松米科技有限公司 | A kind of multimodal data library system, implementation method and server |
| CN109446216A (en) * | 2018-09-12 | 2019-03-08 | 珠海凡泰极客科技有限责任公司 | A kind of message system for supporting SQL syntax |
| CN110333932A (en) * | 2019-06-13 | 2019-10-15 | 上海金融期货信息技术有限公司 | Service orchestration and relationship managing method and system based on container cloud |
| CN111966719A (en) * | 2020-10-21 | 2020-11-20 | 四川新网银行股份有限公司 | Method for refreshing local data cache of distributed consumer credit system in real time |
| CN112532425A (en) * | 2020-11-04 | 2021-03-19 | 浙江大学 | Message routing configuration method facing edge calculation |
| CN112637305A (en) * | 2020-12-16 | 2021-04-09 | 平安消费金融有限公司 | Data storage and query method, device, equipment and medium based on cache |
| CN112650767A (en) * | 2020-11-30 | 2021-04-13 | 中国科学院信息工程研究所 | Data exchange method and system with data filtering preposition |
-
2021
- 2021-06-23 CN CN202110702160.0A patent/CN113486099A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107229639A (en) * | 2016-03-24 | 2017-10-03 | 上海宝信软件股份有限公司 | The storage system of distributing real-time data bank |
| CN109241054A (en) * | 2018-08-02 | 2019-01-18 | 成都松米科技有限公司 | A kind of multimodal data library system, implementation method and server |
| CN109446216A (en) * | 2018-09-12 | 2019-03-08 | 珠海凡泰极客科技有限责任公司 | A kind of message system for supporting SQL syntax |
| CN110333932A (en) * | 2019-06-13 | 2019-10-15 | 上海金融期货信息技术有限公司 | Service orchestration and relationship managing method and system based on container cloud |
| CN111966719A (en) * | 2020-10-21 | 2020-11-20 | 四川新网银行股份有限公司 | Method for refreshing local data cache of distributed consumer credit system in real time |
| CN112532425A (en) * | 2020-11-04 | 2021-03-19 | 浙江大学 | Message routing configuration method facing edge calculation |
| CN112650767A (en) * | 2020-11-30 | 2021-04-13 | 中国科学院信息工程研究所 | Data exchange method and system with data filtering preposition |
| CN112637305A (en) * | 2020-12-16 | 2021-04-09 | 平安消费金融有限公司 | Data storage and query method, device, equipment and medium based on cache |
Non-Patent Citations (2)
| Title |
|---|
| 任友理: "《《大数据技术与应用》大数据技术与应用》", 西北工业大学出版社, pages: 121 - 124 * |
| 刘继刚: "海量高性能分布式消息系统的设计与实现", 《中国优秀硕士学位论文全文数据库信息科技辑》, no. 05, pages 13 - 81 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115665191A (en) * | 2022-10-09 | 2023-01-31 | 浪潮云信息技术股份公司 | User information synchronization method and system based on cloud storage system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7546335B2 (en) | System and method for a data protocol layer and the transfer of data objects using the data protocol layer | |
| US20120323941A1 (en) | Processing Queries for Event Data in a Foreign Representation | |
| US9542708B2 (en) | Event server using caching | |
| WO2016123920A1 (en) | Method and system for achieving integration interface supporting operations of multiple types of databases | |
| US20050015619A1 (en) | Integration infrastrucuture | |
| US20140136717A1 (en) | Configuring cloud resources | |
| US20130166569A1 (en) | Intelligent event query publish and subscribe system | |
| Yongguo et al. | Message-oriented middleware: A review | |
| US10146814B1 (en) | Recommending provisioned throughput capacity for generating a secondary index for an online table | |
| US9201700B2 (en) | Provisioning computer resources on a network | |
| CN113010565B (en) | Server real-time data processing method and system based on server cluster | |
| CN101277304A (en) | Management system and management method for Web service operational environment based on rules | |
| US20090070767A1 (en) | Determining Desired Job Plan Based on Previous Inquiries in a Stream Processing Framework | |
| CN110765165A (en) | Method, device and system for synchronously processing cross-system data | |
| CN113468221A (en) | System integration method based on kafka message data bus | |
| CN111984849A (en) | Information query method, device, equipment and medium | |
| US10445157B2 (en) | Concurrent services caching | |
| US10951540B1 (en) | Capture and execution of provider network tasks | |
| CN113486099A (en) | Memory computing service framework and implementation system | |
| CN112181950B (en) | Construction method of distributed object database | |
| CN111161818A (en) | Medical data exchange sharing system and method based on big data technology | |
| Suzumura et al. | StreamWeb: Real-time web monitoring with stream computing | |
| Jacobs et al. | Bad to the bone: Big active data at its core | |
| CN113051244B (en) | Data access method and device, and data acquisition method and device | |
| Campbell | Service oriented database architecture: App server-lite? |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |