CN108681578B - Business data storage method, device, server and storage medium - Google Patents
Business data storage method, device, server and storage medium Download PDFInfo
- Publication number
- CN108681578B CN108681578B CN201810444053.0A CN201810444053A CN108681578B CN 108681578 B CN108681578 B CN 108681578B CN 201810444053 A CN201810444053 A CN 201810444053A CN 108681578 B CN108681578 B CN 108681578B
- Authority
- CN
- China
- Prior art keywords
- storage
- data
- service
- instance
- service data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Abstract
The invention discloses a business data storage method, a device, a server and a storage medium; the method comprises the following steps: acquiring a public storage instance and a specific storage instance; acquiring a storage instance mapping relation table; the mapping relation table comprises a mapping relation between the public service message and the public storage instance and a mapping relation between the specific service message and the specific storage instance; receiving a service data storage request; determining that the service data belongs to a public service message or a specific service message according to the data characteristics of the service data with the storage request; determining a storage example corresponding to the service data according to the mapping relation table; and storing the service data in the storage instance. The invention not only can realize data storage efficiently according to the requirement of the service data, but also can utilize the hardware characteristics of the data storage server as much as possible.
Description
Technical Field
The present invention relates to the field of data storage technologies, and in particular, to a method, an apparatus, a server, and a storage medium for storing service data.
Background
In the distributed massive message middleware system, a message storage server is a core component of the message middleware; among them, the data storage scheme is a very important ring of the message storage server, and it is directly related to the functions and performances that the whole system can provide.
In the existing mainstream scheme for storing the data of the service messages, the service messages are stored in the storage instance according to the mixed theme, or the service messages are stored in the storage instance according to the independent theme, which cannot realize the storage of the service data efficiently, orderly and customizably. Further, there is a problem that the performance of the storage server is degraded as the number of themes and the number of partitions increase and as the amount of concurrency increases. In addition, the storage of the storage instance in the system to the disk also belongs to a fixed disk-dropping mode, and the storage or disk-dropping mode is single, so that the hardware performance of the storage server cannot be well utilized. Therefore, there is a need to provide a more efficient scheme for the storage of traffic data.
Disclosure of Invention
In order to solve the above technical problem, the present invention provides a method, an apparatus, a server and a storage medium for storing service data, specifically:
in one aspect, a method for storing service data is provided, where the method includes:
acquiring a public storage instance and a specific storage instance;
acquiring a storage instance mapping relation table; the mapping relation table comprises a mapping relation between the public service message and the public storage instance and a mapping relation between the specific service message and the specific storage instance;
receiving a service data storage request;
determining that the service data belongs to a public service message or a specific service message according to the data characteristics of the service data with the storage request;
determining a storage example corresponding to the service data according to the mapping relation table;
and storing the service data in the storage instance.
In another aspect, a service data storage apparatus is provided, the apparatus including:
the storage instance acquisition module is used for acquiring a public storage instance and a specific storage instance;
the mapping relation table acquisition module is used for acquiring a mapping relation table of a storage instance; the mapping relation table comprises a mapping relation between the public service message and the public storage instance and a mapping relation between the specific service message and the specific storage instance;
a storage request receiving module, configured to receive a storage request for service data;
the service message determining module is used for determining that the service data belongs to a public service message or a specific service message according to the data characteristics of the service data with the storage request;
a storage instance determining module, configured to determine, according to the mapping relationship table, a storage instance corresponding to the service data;
and the data storage module is used for storing the service data with the storage request in the storage instance.
Another aspect provides a server, including a processor and a memory, where the memory stores at least one instruction, at least one program, a set of codes, or a set of instructions, and the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by the processor to implement the business data storage method of the above aspect.
Another aspect provides a computer-readable storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by a processor to implement the business data storage method of the above aspect.
The service data storage method, the device, the server and the storage medium provided by the invention have the following beneficial effects:
the data storage examples in the invention comprise public storage examples and specific storage examples; the mapping relation table utilizes the data characteristics of the service messages to flexibly allocate corresponding storage examples for different service messages in a configurable way; when a storage request of a service message sent by a production end is received, the data storage server can find the storage instance corresponding to the service message according to the mapping relation table, and further store the data of the service message in the corresponding storage instance, so that ordered and efficient storage of the data is realized, and the reliability of data storage is improved.
The business data storage method provided by the invention belongs to a universal data processing rule, can not only realize data storage efficiently according to the requirement of business messages, but also utilize the hardware characteristics of a data storage server as much as possible; moreover, on the basis of realizing the high-efficiency storage of the data of the service message, the method further provides convenience for the consumption of the service message by the subsequent consumption end; the adaptability and the processing capacity of the data storage server to the service data in the data persistence processing are improved, and the processing cost is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions and advantages of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of a business data storage method provided in an embodiment of the present specification;
fig. 2 is a schematic structural diagram of a service data storage scheme provided by an embodiment of the present specification;
fig. 3 is a flowchart of another business data storage method provided by the embodiments of the present specification;
fig. 4 is a flowchart of another service data storage method provided in the embodiments of the present specification;
fig. 5 is a schematic structural diagram of a storage example in a business data storage scheme provided in an embodiment of the present specification;
fig. 6 is a flowchart of another service data storage method provided in the embodiments of the present specification;
FIG. 7 is a block diagram of a business data storage device provided by an embodiment of the present disclosure;
FIG. 8 is a block diagram of another service data storage device provided by embodiments of the present description;
FIG. 9 is a block diagram of another service data storage device provided by embodiments of the present description;
fig. 10 is a schematic structural diagram of a service data storage server provided in an embodiment of the present specification.
Detailed Description
The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In the existing scheme that an MQ (message queue) message queue stores file (service message) data on the ground, some files are gradually increased along with the increase of the number of Topic and the number of part; the time of disk IO (read/write) and disk util (the operation time ratio for IO in the period) is also increased, the read/write performance is reduced, and the access flow of the storage server is reduced. Some of the files form a large number of small files along with the increase of Topic and partition, so that the performance of the disk cannot be effectively improved; meanwhile, the custom processing cannot be performed on the Topic services with different data characteristics (such as data aging requirements and different data disk refreshing requirements). Still some MQ message queues cannot take advantage of the performance advantages of the storage server in the context of multiple disk storage servers.
In order to solve the technical problems, the invention provides a new service data storage scheme, which belongs to the improvement of the message storage scheme in the existing MQ message queue, provides a scheme of constructing a storage instance unit and carrying out parameterization setting according to different service characteristics, different performance requirements and the number of disk blocks, and solves the problem of the existing MQ message queue storage on a high-concurrency and large-size storage server; the scheme supports mixed storage and independent storage, and the storage position can be independently specified according to the characteristics of server hardware. Specifically, the method comprises the following steps:
an embodiment of the present specification provides a method for storing business data, and fig. 1 is a schematic flow diagram of a method for storing business data provided by an embodiment of the present specification; it is noted that the present specification provides the method steps as described in the examples or flowcharts, but may include more or less steps based on conventional or non-inventive labor. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. In practice, the system or server product may be implemented in a sequential or parallel manner (e.g., parallel processor or multi-threaded environment) according to the embodiments or methods shown in the figures. Specifically, as shown in fig. 1, the method includes:
s101, acquiring a public storage instance and a specific storage instance;
in step S101, the established storage instance is established by a system administrator or an operation and maintenance person of the data storage server based on the data storage server platform according to the service requirement; thereby causing the data storage server to obtain the storage instance.
As shown in fig. 2, each storage instance includes one Data (Data) document and one Index (Index) document; the Data document stores original message (service Data sent by a production end) Data received by the Data storage server; the Index document stores message Index data (message attribute information) of the original message, such as Topic, partition, received time, and the like; to associate the information items of the original message in the Data document.
In addition, it is explained that the service characteristics of the service data in the same storage instance need to meet consistent requirements, for example, the service characteristics of the service data in the same storage instance need to be less than or equal to a preset service characteristic; specifically, the service features include: aging period, frequency refreshing strategy (refreshing frequency of disk dropping), storage strategy (disk position stored in a storage server), whether isolation is required, and the like; for example, if the aging period in the preset service characteristics is 5 days, two services need to be stored, and if the aging period of one service is 2 days, the service meets the storage requirement of the storage instance; the aging period of the other service is 7 days, the storage requirement of the storage instance is not met.
Because the Topic of the physical service message may contain a plurality of partition partitions, each partition corresponds to a document for storing all messages and indexes corresponding to the partition; by classifying the service features of the service data into groups according to preset conditions for storage, the problem that the processing performance of the server is affected due to more documents when the quantity of Topic and partition to which the service data belongs is large can be solved.
As shown in fig. 2, the storage instances include a common storage instance and a specific storage instance in a parallel arrangement. Wherein, the public storage instance and the specific storage instance are explained as follows:
the service data is stored in the common storage instance of the storage server by default, while the embodiment gives a different storage scheme from the common service for specific service data, and constructs a new storage instance (specific storage instance) according to service attributes (service characteristics), so that the specific production service is stored in the appropriate specific storage instance. Through the classification of the common storage instances and the specific storage instances, the data storage server meets the storage requirements of different services, and the storage performance of the storage server is exerted to the greatest extent.
The service data stored in the public storage instance and the specific storage instance support the real-time allocation and data management of a system responsible person or operation and maintenance personnel, and the management can be carried out according to the allocation; the process of constructing the storage instance is a definition process of the storage instance according to the configuration by the program.
In detail, the common storage instance is configured to store data of a common service message, for example, a service message corresponding to a journal; or the service characteristics conform to the service messages defined by the default of the system, for the service messages defined by the default, for example, the system sets a default 3-day data aging period, a low-frequency second-level data-landing refreshing frequency of 1 second and the like, the data volume is in the billions/day, the peak value is in the 1 ten thousand/second and the like, and the service messages set by the default are satisfied.
In detail, the specific storage instance is set to be used for storing data of specific service information, such as service data which is required to be read from time to time, or service messages which relate to sensitive words such as financial transfers and the like; or service messages with service characteristics that the aging period is 7 days (larger than the default aging period of 3 days), the refreshing frequency of data landing is 10ms and the like, and the frequency is higher than that of millisecond-level service messages.
For example: if the data of the service A is applied for storage; the service attribute or service characteristic of the service a is required to be an aging period of 3 days, a disk refreshing frequency of 10 seconds, and the number of the service attributes or service characteristics is about 1 hundred million per day; the system responsible person or operation and maintenance person puts the service data into the public storage examples [ STORE-PUB-1 and STORE-PUB-2 ] according to the service characteristics; and assigns a Topic name as Topic-a to service a and further STOREs it in a corresponding memory block, such as memory block STORE-PUB-1.
If the data of the service B is applied for storage; the service attribute or service characteristic of the service B is required to be large data volume (2 billion data volume per day), the aging period is required to be 10 days, and the flash frequency is required to be 500 pieces; the system leader or operation and maintenance personnel put the service data into storage examples (STORE-Topic-B-1, STORE-Topic-B-2 and STORE-Topic-B-3) specially allocated for the service B according to the service characteristics; and is named Topic-b according to the assigned Topic name for service b and further stored in a corresponding memory block, such as memory block STORE-Topic-b-3.
After the storage allocation of the service a and the service B is applied, when the service a or the service B comes to produce and consume, the production and consumption of the service data can be performed, specifically:
a, producing a service, wherein a data storage server finds a storage example [ STORE-PUB-1, STORE-PUB-2 ] according to the contrast relation between the data characteristics of the service data and the storage example after receiving the produced service data; further, the transmitted production service data carries a Topic name Topic-a, a storage block STORE-PUB-1 is selected according to the Topic name Topic-a, and the data is stored in the storage block.
And B, in the service production, after the data storage server receives the produced service data, finding out storage examples [ STORE-Topic-B-1, STORE-Topic-B-2 and STORE-Topic-B-3 ] according to the comparison relationship between the data characteristics of the service data and the storage examples, further, sending the production service data carrying a Topic name Topic-B, and selecting a storage block STORE-Topic-B-3 according to the Topic name Topic-B to STORE the data.
When the service A is consumed, a program of the service A initiates a registration request of a storage server, so that a consumption target service A carries Topic-A to register, and the storage server finds a storage example (STORE-PUB-1, STORE-PUB-2) corresponding to the Topic-a according to the received registration information and allocates a storage block for the storage example; and then when the A service initiates a getMessage consumption request, consuming and processing data in the storage block with the Topic name being Topic-a corresponding to the two storage blocks.
S102, obtaining a storage instance mapping relation table; the mapping relation table comprises a mapping relation between the public service message and the public storage instance and a mapping relation between the specific service message and the specific storage instance;
in step S102, the establishment of the mapping relationship between the service message and the storage instance is performed by a system administrator or an operation and maintenance person based on the platform of the data storage server, so that the data storage server obtains the mapping relationship table, where the mapping relationship table belongs to a static mapping result.
The business messages are divided into a plurality of classes according to different data properties, each class of messages applies for a single Topic, correspondingly, the whole business data can have one or more Topic topics, and the business data of different Topic topics are stored in different storage examples according to business characteristics; therefore, the comparison relationship between the service message and the storage instance is one-to-one or one-to-many, that is, the data of one service message can be stored in one storage instance or in a plurality of storage instances. Wherein, the data of the multiple service messages can also only apply for one Topic, and are distinguished by different characteristics carried in the messages; therefore, the contrast relationship between the service data and the storage instance can also be many-to-one, that is, a plurality of service data are stored in one storage instance.
S103, receiving a storage request of service data;
specifically, the data object for storing is service data sent by a Producer (message Producer for producing service messages); in step S103, the Producer sends the produced service message to the data storage server, and the data storage server receives the data storage request of the service message.
S104, determining that the service data belongs to a public service message or a specific service message according to the data characteristics of the service data with the storage request;
the public service message comprises a service message corresponding to the journal or a service message defined by local default, and the like; the specific service message comprises a service message related to virtual article transaction and the like. The data characteristics of the service data at least comprise data capacity, data aging period, data flash frequency and the like. Wherein, the frequency of the disk brushing is related to the importance degree of the data.
After receiving a storage request of the service data, judging the data characteristics of the service data, and determining whether the service data belongs to a public service message or a specific service message according to a judgment result;
s105, determining a storage example corresponding to the business data according to the mapping relation table;
step S105 belongs to the logic path for searching the storage example corresponding to the service message; the data storage server is used for storing the mapping relation table between the instance and the service message according to the established storage instance; after receiving the service message sent by the production end, finding the storage instance corresponding to the currently received service message by searching the mapping relation table.
It should be noted that, according to the defined contrast relationship between the service message and the storage instance, in the subsequent operation process, as long as the contrast relationship is not changed, the previously defined storage area will be directly accessed when the service message is produced or consumed.
The storage allocation rule of the storage instance corresponding to the service message may also include a designated allocation rule or a non-designated allocation rule. Specifically, a suitable storage instance is configured according to the requirements of the production service, belonging to a specified allocation rule. For the non-specified allocation rule, for example, data is stored in a polling mode, and after the service message is received, the storage instances sequentially store the data; each storage instance is possible to store the data of the service message sent by the production end, so that the load balance is satisfied.
And S106, storing the service data with the storage request in the storage instance.
In step S106, after obtaining the storage instance where the service data sent by the sending end is located according to the above determination, the service data is stored in the corresponding storage instance.
The embodiment of the present specification further provides a service data storage method, where it is to be explained that the common storage example in the embodiment includes a multi-topic storage block for storing service data with different topic characteristics. The specific storage instance comprises a single theme storage block used for storing the business data with the same theme characteristics; therefore, accordingly: as shown in fig. 3, a corresponding storage instance is determined from the mapping relation table in step S104, and the service data with the storage request is stored in the storage instance in step S105; then also comprises the following steps:
s107, determining a storage block corresponding to the business data according to the theme of the business data; and storing the service data in the storage block.
In the embodiment, the data storage server allocates different storage instances for different service data by using the characteristics of production services through the established storage instances and the set mapping relation table; when a storage request of the service data sent by the production end is received, the storage instance corresponding to the service data is searched, and the service data can be further written into the storage block corresponding to the storage instance according to the theme of the service data, so that the service data is stored in a customized and efficient manner, and the hardware characteristics of the data storage server are utilized to the greatest extent.
An embodiment of the present specification further provides a service data storage method, which belongs to a scheme that a message middleware adopts a disk as data persistence, and specifically, as shown in fig. 4, the method includes:
s201, acquiring a public storage instance and a specific storage instance;
based on a platform of the data storage server, a system principal or an operation and maintenance person constructs a new storage instance model according to the requirements of production business messages, wherein the storage instances in the model comprise a public storage instance and a specific storage instance which are arranged in parallel.
It should be noted that the common storage instance includes a multi-Topic storage block, as shown in fig. 5, that is, a hybrid storage instance, for storing business data with different Topic features, such as containing Topic1, Topic2, Topic3, etc. in fig. 5. The specific storage instance comprises a single subject storage block, as shown in fig. 5, that is, a single Topic storage instance, for storing business data having the same subject characteristics, such as only Topic5 in fig. 5.
The service characteristics of the service messages stored in the mixed storage instance are stored under the condition that the minimum storage requirement of the storage instance is met. Specifically, when the capacity of the service data is less than or equal to a preset capacity, the aging period of the service data is less than or equal to a preset aging period, and the frequency of flushing the service data is less than or equal to a preset frequency of flushing the service data; the service data belongs to a common service message. When the capacity of the service data is larger than a preset capacity, or the aging period of the service data is larger than a preset aging period, or the frequency of the service data being refreshed is larger than a preset refreshing frequency; the service data belongs to specific service data.
For the specific service Data, the grade PBDL (parametric Business Data level) of the specific service Data can be divided according to the comparison result of the parameters in the Data characteristics and the preset parameters; and storing the specific service data in the corresponding specific storage instance according to the level to which the specific service data belongs.
For example, the following steps are carried out: judging the level of the specific service by analyzing the multiple relation between the capacity (Curr _ capa) of the service data and the preset capacity (Pre _ capa); for example, the first capacity multiple a1 is 100, and the second capacity multiple a2 is 300;
if Curr _ capa > a1 Pre _ capa, belonging to A1 level specific service data;
if Curr _ capa > a2 Pre _ capa, it belongs to a2 level specific service data.
Or, judging the level of the specific service by analyzing the multiple relation between the aging period (Curr _ period) of the service data and the preset aging period (Pre _ period); for example, the first aging period multiple b1 is 2, and the second aging period multiple b2 is 4;
if Curr _ period > B1 Pre _ period, belonging to B1 level specific service data;
if Curr _ period > B2 Pre _ period, then it belongs to B2 level specific service data.
Or, judging the level of the specific service by analyzing the multiple relation between the frequency (Curr _ freq) of the service data and the preset frequency (Pre _ freq) of the service data; for example, the first brush-tray frequency multiple b1 is 2, the second brush-tray frequency multiple b2 is 4c1 is 10, and c2 is 100;
if Curr _ freq > C1 Pre _ freq, belonging to C1 level specific service data;
if Curr _ freq > C2 Pre _ freq, then it belongs to C2 level specific service data.
Or, when the capacity (Curr _ capa) of the service data is greater than a preset capacity (Pre _ capa), the aging period (Curr _ period) of the service data is greater than a preset aging period (Pre _ period), and the flash frequency (Curr _ freq) of the service data is greater than a preset flash frequency (Pre _ freq), determining the level to which the PBDL specific service data belongs according to a formula calculation result:
PBDL＝0.6*L_period+0.1*L_capa+0.3*L_freq。
wherein, L _ period is the level to which the aging period of the service data is judged, L _ capa is the level to which the capacity of the service data is judged, and L _ freq is the level to which the flash frequency of the service data is judged.
It should be noted that the aging period is not a necessary requirement for distinguishing whether new storage instances are divided, and this can be arranged reasonably according to the resource situation and the service importance degree.
If the data aging period does not meet the requirement and no proper storage instance exists, a new storage instance can be reestablished for the service data, and a corresponding relation is established; in addition, the aging period in the storage instance for storing the old aging period can be directly set to be a long aging period by operation and maintenance personnel without new construction so as to meet the requirement of the current service message; in this way, the arrangement is reasonable.
For example, the data in one memory block of the mixed memory example is brushed once every 1000 seconds or 10 seconds, the aging period is 3 days, and the capacity is about 5 billion pieces per day; if the business is an hourly business (data processing is carried out according to the unit of hour), the aging period requires 2 days, and the capacity is only 5 hundred million pieces per day; if the data is a minute service (data processing is carried out according to the unit of minute), the aging period is 3 days, and the quantity is 500 hundred million, and the data can be placed in a mixed storage example. However, if some services require milliseconds to be accessed, the aging period is 5 days, and the capacity is only 10 hundred million per day; however, since the aging period of 5 days is more than 3 days of the storage requirement, the service does not meet the minimum requirement of the mixed storage instance, and a mixed storage instance is newly opened. Wherein, different business data comprise different themes (Topic), and different business messages carry the specified theme when the data are produced or consumed; when a business message sent by a production end is received, the data storage server can find the storage block where the business message is located according to the Topic (Topic) of the business message to realize storage. And when the service message needs to be consumed, the storage block where the service message is located can be searched in a targeted manner according to the Topic (Topic) corresponding to the service message, so that the processing efficiency is improved, and the processing cost is reduced.
S202, acquiring a storage instance mapping relation table; the mapping relation table comprises a mapping relation between the public service message and the public storage instance and a mapping relation between the specific service message and the specific storage instance;
specifically, based on the platform of the data storage server, a system administrator or operation and maintenance personnel establishes a mapping relationship between a service message and a storage instance according to the requirements of a production service, so that the data storage server obtains the mapping relationship table.
S203, receiving a storage request of service data;
the production end sends the produced business information to the data storage server, and the data storage server receives the data storage request of the business information.
S204, determining that the service data belongs to a public service message or a specific service message according to the data characteristics of the service data with the storage request;
in detail, after receiving the service data sent by the production end, the data storage server analyzes the characteristics of the service data and determines that the service message to which the service data belongs is a public service message or a specific service message.
S205, determining a storage example corresponding to the service data according to the mapping relation table;
in detail, after the data storage server analyzes the characteristics of the service data to determine the corresponding service message, the data storage server finds out the storage instance required to be stored by the currently received service message by searching the mapping relation table.
S206, storing the service data in the storage instance.
And after judging the corresponding storage instance of the received service data, the data storage server stores the current service data in the storage instance.
Further, storing the service data with the storage request in the storage instance at step S206, which then includes:
and S207, storing the storage instance storing the service data in a specified disk.
After the service data is stored in the storage instance, as shown in fig. 2, the storage instance storing the service data is further landed in the disks of the local disk group of the data storage server, such as disks C1, C2, C3, and C4 in the figure. Moreover, the storage instances are stored in the appointed corresponding disks, and real-time allocation of a system responsible person or operation and maintenance personnel is supported; the method and the system can manage the disk dropping of the storage instances according to the configuration requirements, and specify the disks stored in different positions for different storage instances, thereby improving the parallel processing capacity of the data storage server.
The data storage method provided by the embodiment belongs to a universal processing rule, can efficiently realize data storage according to the requirement of the produced business data, and can utilize the hardware characteristics of the data storage server as much as possible, thereby improving the adaptability of the data storage server to the business data in the data persistence processing and reducing the processing cost. Furthermore, in this embodiment, the storage of the service data may be dynamically specified according to the drop of the storage instance, and the storage and the drop can be performed in a configurable manner, so that the parallel processing capability of the data storage server is improved.
An embodiment of the present specification further provides a service data storage method, as shown in fig. 6, where the method includes:
s301, acquiring a public storage instance and a specific storage instance;
in the public storage instance, data characteristics (data aging period, disk refreshing strategy and the like) of general service data are defined, a comparison relation between the service data meeting the strategy and the public storage instance is established, and the service data is routed to the public storage instance for processing during production and consumption. For a specific service, for example, when the message volume is very large, or the data characteristics (flash strategy, aging period, etc.) of service data have special requirements but the common storage instance cannot meet the requirements, in the data storage server, a specific storage instance for the specific service is constructed at a position parallel to the common storage instance, and a contrast relationship between the service and the specific storage instance is established, and the service data is routed to the specific storage instance for processing when being produced and consumed.
It should be noted that, if a situation that the amount of data written in a certain storage instance is large and performance needs to be improved is found, multiple storage instances may be constructed for the service data corresponding to the storage instance, a new storage instance is constructed in parallel on the basis of an existing storage instance and added into the comparison relationship between the service data and the storage instances, and the data is routed to a common storage real-time routine for processing when being produced and consumed. That is to say, the common storage instance and the specific storage instance can be configured into a plurality of instances according to the data volume received by the data storage server, so as to improve the concurrent processing capacity of the data storage server on the data.
S302, obtaining a storage instance mapping relation table; the mapping relation table comprises a mapping relation between the public service message and the public storage instance and a mapping relation between the specific service message and the specific storage instance;
in step S302, after the common storage instance or the specific storage instance is constructed, a comparison relationship between the service data and the storage instance is established; and, when the storage instances are expanded into a plurality of instances, a contrast relationship between the service data and the plurality of storage instances is also established to form a mapping relationship table.
For example, if the data volume of the service data corresponding to Topic-a is found to increase and the service corresponding to Topic-a needs to be stored by the reconstructed storage instance alone, a system leader or operation and maintenance staff may add a new storage instance C to the service corresponding to Topic-a according to an interface of the data storage server, and after configuration becomes effective, the service data may be dropped onto the C storage instance when a storage request of the service data is subsequently sent; that is, after the contrast relationship is transferred, as long as the system administrator or operation and maintenance personnel does not modify the contrast relationship, the subsequent production and consumption of the business data will operate on the C storage instance.
S303, receiving a storage request of service data;
the production end sends the produced service information to the data storage server, and the data storage server receives the storage request of the service data.
S304, determining that the service data belongs to a public service message or a specific service message according to the data characteristics of the service data with the storage request;
in detail, after receiving a service message sent by a production end, a data storage server analyzes data characteristics of the service message and determines a service message type to which the service data belongs.
S305, determining a storage example corresponding to the business data according to the mapping relation table
And finding out the storage example which needs to be stored by the currently received service message by searching the mapping relation table. And further determining the corresponding storage block according to the theme of the service message.
S306, storing the service data with the storage request in the storage instance.
Specifically, after receiving the service data sent by the production end, the data storage server finds a storage instance in which the currently received service data needs to be stored by searching the mapping relation table, and stores the storage instance in the storage instance. And further storing the service data in the storage block based on the determined storage block.
Further, the method in this embodiment further includes:
s307, receiving a consumption request for consuming the service data;
and the Customer at the consumption end sends a consumption request of the produced service data to the data storage server, and the data storage server receives the consumption request.
S308, determining a storage block stored in the business data according to the theme of the business data;
wherein, different business data have different themes (Topic); at the time of data consumption, the data storage server can find the storage instance where the business data is located according to the label (theme) of the business data to be consumed.
S309, sending the service data stored in the storage block to enable the service data to be consumed.
In step S309, after finding the storage instance of the service data to be consumed, the data storage server sends the service data in the storage instance to the data consuming side, so that the consuming side reads the service data which is sent by the production side and stored locally in the data storage server, and consumes the service data. The consumption of data refers to acquiring data sent by a production end from a storage area of a Topic specified by a storage server, for example, the production end sends three data packets, namely data a, data b and data c, to the Topic-a, and a consumer subscribing to the Topic-a can acquire the corresponding three data packets from the storage server.
The embodiment performs independent storage setting aiming at different service storage requirements, different server configurations and performance requirements, and gives play to the server performance to the greatest extent; and on the basis of realizing the efficient storage of the service data, convenience is further provided for the subsequent consumption end to sequentially consume the service data.
An embodiment of the present specification further provides a service data storage device, as shown in fig. 7, the device includes:
a storage instance obtaining module 110, configured to obtain a common storage instance and a specific storage instance;
a mapping relation table obtaining module 120, configured to obtain a mapping relation table of a storage instance; the mapping relation table comprises a mapping relation between the public service message and the public storage instance and a mapping relation between the specific service message and the specific storage instance;
a storage request receiving module 130, configured to receive a storage request of service data;
a service message determining module 140, configured to determine that the service data belongs to a common service message or a specific service message according to a data feature of the service data having the storage request;
a storage instance determining module 150, configured to determine, according to the mapping relationship table, a storage instance corresponding to the service data;
a data storage module 160, configured to store the service data with the storage request in the storage block.
The public storage instance comprises a multi-theme storage block used for storing business data of a plurality of different themes;
the specific storage example comprises a single theme storage block used for storing the business data of the same theme.
Specifically, the storage instance determining module 150 includes:
and a storage block determining unit 151, configured to determine, according to the theme of the service data, a storage block corresponding to the service data.
An embodiment of the present specification further provides a service data storage device, as shown in fig. 8, where the device includes:
a storage instance obtaining module 210, configured to obtain a common storage instance and a specific storage instance;
a mapping relation table obtaining module 220, configured to obtain a mapping relation table of a storage instance; the mapping relation table comprises a mapping relation between the public service message and the public storage instance and a mapping relation between the specific service message and the specific storage instance;
a storage request receiving module 230, configured to receive a storage request of service data;
a service message determining module 240, configured to determine that the service data belongs to a common service message or a specific service message according to a data characteristic of the service data having the storage request;
a storage instance determining module 250, configured to determine, according to the mapping relationship table, a storage instance corresponding to the service data;
and the data storage module 260 is used for storing the service data with the storage request in the storage instance.
The device further comprises:
and a storage instance saving module 270, configured to save the storage instance storing the service data in a specified disk.
An embodiment of the present specification further provides a service data storage device, as shown in fig. 9, where the device includes:
a storage instance obtaining module 310, configured to obtain a common storage instance and a specific storage instance;
a mapping relation table obtaining module 320, configured to obtain a mapping relation table of a storage instance; the mapping relation table comprises a mapping relation between the public service message and the public storage instance and a mapping relation between the specific service message and the specific storage instance;
a storage request receiving module 330, configured to receive a storage request of service data;
a service message determining module 340, configured to determine, according to a data feature of service data with a storage request, that the service data belongs to a common service message or a specific service message;
a storage instance determining module 350, configured to determine, according to the mapping relationship table, a storage instance corresponding to the service data;
a data storage module 360, configured to store the service data with the storage request in the storage instance.
The device further comprises:
a consumption request obtaining module 370, configured to receive a consumption request of the service data;
a data storage determining module 380, configured to determine, according to the theme of the service data, a storage block in which the service data is stored;
a data sending module 390, configured to send the service data stored in the storage block, so that the service data is consumed.
The embodiment of the present specification provides a business data storage server, where the parameter identification server includes a processor and a memory, where the memory stores at least one instruction, at least one program, a code set, or a set of instructions, and the at least one instruction, the at least one program, the code set, or the set of instructions is loaded and executed by the processor to implement the business data storage method provided in the above method embodiment.
The memory may be used to store software programs and modules, and the processor may execute various functional applications and data processing by operating the software programs and modules stored in the memory. The memory can mainly comprise a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs needed by functions and the like; the storage data area may store data created according to use of the apparatus, and the like. Further, the memory may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory may also include a memory controller to provide the processor access to the memory.
The embodiment of this specification further provides a schematic structural diagram of a server, please refer to fig. 10, where the server 1200 is configured to implement the service data storage method provided in the foregoing embodiment, specifically, the server structure may include the service data storage device. The server 1200 may vary significantly depending on configuration or performance, and may include one or more Central Processing Units (CPUs) 1210 (e.g., one or more processors) and memory 1230, one or more storage media 1220 (e.g., one or more mass storage devices) that store applications 1223 or data 1222. Memory 1230 and storage media 1220, among other things, may be transient storage or persistent storage. The program stored in the storage medium 1220 may include one or more modules, each of which may include a series of instruction operations for a server. Further, the central processor 1210 may be configured to communicate with the storage medium 1220, and execute a series of instruction operations in the storage medium 1220 on the server 1200. The server 1200 may also include one or more power supplies 1260, one or more wired or wireless network interfaces 1250, one or more input-output interfaces 1240, and/or one or more operating systems 1221, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, and so forth.
The present specification further provides a storage medium, which may be disposed in a server to store at least one instruction, at least one program, a code set, or a set of instructions related to implementing a business data storage method in the method embodiment, where the at least one instruction, the at least one program, the code set, or the set of instructions is loaded and executed by the processor to implement the business data storage method provided by the above method embodiment.
Alternatively, in this embodiment, the storage medium may be located in at least one network server of a plurality of network servers of a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
An embodiment of the present specification further provides a server, where the server includes a processor and a memory, where the memory stores at least one instruction, at least one program, a code set, or a set of instructions, and the at least one instruction, the at least one program, the code set, or the set of instructions is loaded and executed by the processor to implement the business data storage method according to the method embodiment; the at least one instruction, the at least one program, the set of codes, or the set of instructions comprises instructions for:
acquiring a public storage instance and a specific storage instance;
acquiring a storage instance mapping relation table; the mapping relation table comprises a mapping relation between the public service message and the public storage instance and a mapping relation between the specific service message and the specific storage instance;
receiving a service data storage request;
determining that the service data belongs to a public service message or a specific service message according to the data characteristics of the service data with the storage request;
determining a storage example corresponding to the service data according to the mapping relation table;
and storing the service data in the storage instance.
Further, the storing the service data in the storage instance, then comprises:
and storing the storage instance storing the service data in a specified disk.
The public storage instance comprises a multi-theme storage block used for storing business data of a plurality of different themes;
the specific storage example comprises a single theme storage block used for storing the business data of the same theme.
Further, the storage instance corresponding to the service data is determined according to the mapping relation table; the method comprises the following steps:
and determining a storage block corresponding to the business data according to the theme of the business data.
Wherein the method further comprises:
receiving a consumption request of service data;
determining a storage block stored in the business data according to the theme of the business data;
and transmitting the service data stored in the storage block to enable the service data to be consumed.
The public service message comprises a service message corresponding to the journal or a service message defined by local default; the specific business message comprises a business message related to virtual article transaction. The data characteristics of the service data at least comprise data capacity, data aging period and data flash frequency.
The present specification also provides a computer readable storage medium, in which at least one instruction, at least one program, a code set, or a set of instructions is stored, and the at least one instruction, the at least one program, the code set, or the set of instructions is loaded and executed by a processor to implement the business data storage method according to the method embodiment; the at least one instruction, the at least one program, the set of codes, or the set of instructions comprises instructions for:
acquiring a public storage instance and a specific storage instance;
acquiring a storage instance mapping relation table; the mapping relation table comprises a mapping relation between the public service message and the public storage instance and a mapping relation between the specific service message and the specific storage instance;
receiving a service data storage request;
determining that the service data belongs to a public service message or a specific service message according to the data characteristics of the service data with the storage request;
determining a storage example corresponding to the service data according to the mapping relation table;
and storing the service data in the storage instance.
Further, the storing the service data in the storage instance, then comprises:
and storing the storage instance storing the service data in a specified disk.
The public storage instance comprises a multi-theme storage block used for storing business data of a plurality of different themes;
the specific storage example comprises a single theme storage block used for storing the business data of the same theme.
Further, the storage instance corresponding to the service data is determined according to the mapping relation table; the method comprises the following steps:
and determining a storage block corresponding to the business data according to the theme of the business data.
Wherein the method further comprises:
receiving a consumption request of service data;
determining a storage block stored in the business data according to the theme of the business data;
and transmitting the service data stored in the storage block to enable the service data to be consumed.
The public service message comprises a service message corresponding to the journal or a service message defined by local default; the specific business message comprises a business message related to virtual article transaction. The data characteristics of the service data at least comprise data capacity, data aging period and data flash frequency.
The invention provides the data characteristics of the service message, such as data capacity, data landing frequency, data aging period and the like; finding out a proper storage example for the service message sent by the sending end according to the defined mapping relation table; if the data aging period does not meet the requirement and no proper storage instance exists, a new storage instance can be reestablished for the service data, and a corresponding relation is established; in addition, there is also a case that the new storage is not created, and the operation and maintenance personnel directly set the aging period in the storage instance for storing the old aging period to be a long aging period, so as to meet the requirement of the current service message and further realize flexible storage.
When a production end requests to store a produced service message, a Topic name Topic-a of the service data and data information of the service message to be stored are carried in request information; and the production terminal produces the service message to the storage server of the MQ message queue, and after receiving the production message, the storage server of the MQ message queue finds the corresponding storage block according to the Topic name Topic-a, stores the data and returns a storage result to the production terminal.
When a consumer applies for a consumption request of a business message, carrying a Topic name Topic-a of the business message to be consumed, a consumption group name carried by a sent request message, and data information of the business message to be consumed; and after receiving the consumption request, the storage server of the message queue finds the corresponding storage block according to the Topic name Topic-a carried in the consumption request, and performs data consumption processing.
The system responsible person or operation and maintenance person can monitor the progress of the storage block regularly according to the production or consumption service data, and perform the capacity expansion or contraction of the storage instance and the migration operation of Topic.
The invention can lead the system responsible person or the operation and maintenance person to find out the proper storage example for the service according to the data requirement of the service message, and can reestablish the new storage example if no proper storage example exists. The new storage instance is established by issuing an operation request to a storage server of the MQ message queue by a system leader or operation and maintenance personnel. The whole storage scheme of the service data can flexibly allocate the storage blocks according to the condition of the service data, further flexibly configure the disks for the storage examples, can meet the requirements of more service data, and further improve the processing efficiency of the corresponding server and system and the user viscosity.
It is explained that the scheme of the invention is applied to the TDTube message storage module of the TDBank business management system, and the part belongs to a complete data storage scheme without a front-end interface.
It should be noted that: the sequence of the embodiments in this specification is merely for description, and does not represent the advantages or disadvantages of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (13)
1. A business data storage method is characterized in that the method is a method for storing business data in data persistence processing for a data storage server in a message middleware system, and the method comprises the following steps:
acquiring a common storage instance for storing data of the common service message or the service message with service characteristics conforming to default definition of a system, and a specific storage instance for storing data of specific service information; the storage example is established based on a data storage server platform according to the requirements of the service; thereby causing the data storage server to obtain the storage instance; the common storage instance and the specific storage instance are arranged in parallel;
acquiring a storage instance mapping relation table; the mapping relation table comprises a mapping relation between the public service message and the public storage instance and a mapping relation between the specific service message and the specific storage instance; the establishment of the mapping relation between the service message and the storage instance is carried out on the basis of the platform of the data storage server, so that the data storage server obtains the mapping relation table, and the mapping relation table belongs to a static mapping result;
receiving a service data storage request, wherein the service data storage request is that a production end sends a produced service message to a data storage server, and the data storage server receives the data storage request of the service message;
determining that the service data belongs to a public service message or a specific service message according to all data characteristics for distinguishing service message types of the service data with the storage request; the data characteristics of the service data comprise data capacity, data aging period and data flash frequency; when the capacity of the service data is less than or equal to a preset capacity, the aging period of the service data is less than or equal to a preset aging period, and the frequency of flushing the service data is less than or equal to a preset frequency of flushing the service data; the service data belongs to a public service message;
when the capacity of the service data is larger than a preset capacity, or the aging period of the service data is larger than a preset aging period, or the frequency of the service data being refreshed is larger than a preset refreshing frequency; the service data belongs to specific service data;
determining whether the storage instance corresponding to the service data is a public storage instance or a specific storage instance according to the mapping relation table;
and storing the service data in the storage example, wherein the data characteristics of the service data in the same storage example meet the requirement of consistency.
2. The business data storage method according to claim 1, wherein said storing said business data in said storage instance, thereafter comprises:
and storing the storage example storing the service data in a specified disk.
3. The business data storage method of claim 1,
the public storage example comprises a multi-theme storage block used for storing business data of a plurality of different themes;
the specific storage example comprises a single theme storage block used for storing the business data of the same theme.
4. The storage method according to claim 3, wherein the storing the service data in the storage instance comprises:
and determining a storage block corresponding to the service data according to the theme of the service data.
5. The business data storage method of claim 3, wherein the method further comprises:
receiving a consumption request of service data;
determining a storage block stored in the business data according to the theme of the business data;
and transmitting the service data stored in the storage block to enable the service data to be consumed.
6. The service data storage method according to claim 1, wherein the common service message comprises a service message defined by a local default; the specific business message comprises a business message related to virtual article transaction.
7. A business data storage apparatus, wherein the apparatus stores business data in a data persistence process for a data storage server in a message middleware system, the apparatus comprising:
a storage instance acquisition module used for acquiring the data used for storing the public service message or the public storage instance of the service message with the service characteristics conforming to the default definition of the system and the specific storage instance of the data used for storing the specific service information; the storage example is established based on a data storage server platform according to the requirements of the service; thereby causing the data storage server to obtain the storage instance; the common storage instance and the specific storage instance are arranged in parallel;
the mapping relation table acquisition module is used for acquiring a mapping relation table of a storage instance; the mapping relation table comprises a mapping relation between the public service message and the public storage instance and a mapping relation between the specific service message and the specific storage instance; the establishment of the mapping relation between the service message and the storage instance is carried out on the basis of the platform of the data storage server, so that the data storage server obtains the mapping relation table, and the mapping relation table belongs to a static mapping result;
the data storage server is used for receiving a data storage request of the service message;
the service message determining module is used for determining that the service data belongs to a public service message or a specific service message according to all data characteristics of the service data with the storage request for distinguishing the service message types; the data characteristics of the service data comprise data capacity, data aging period and data flash frequency; when the capacity of the service data is less than or equal to a preset capacity, the aging period of the service data is less than or equal to a preset aging period, and the frequency of flushing the service data is less than or equal to a preset frequency of flushing the service data; the service data belongs to a public service message;
when the capacity of the service data is larger than a preset capacity, or the aging period of the service data is larger than a preset aging period, or the frequency of the service data being refreshed is larger than a preset refreshing frequency; the service data belongs to specific service data;
a storage instance determining module, configured to determine, according to the mapping relationship table, whether a storage instance corresponding to the service data is a common storage instance or a specific storage instance;
and the data storage module is used for storing the service data with the storage request in the storage instance, and the data characteristics of the service data in the same storage instance meet the requirement of consistency.
8. The business data storage device of claim 7, wherein said device further comprises:
and the storage instance storage module is used for storing the storage instance storing the service data in a specified disk.
9. The service data storage device of claim 7,
the public storage example comprises a multi-theme storage block used for storing business data of a plurality of different themes;
the specific storage example comprises a single theme storage block used for storing the business data of the same theme.
10. The business data storage device of claim 9 wherein said storage instance determination module comprises:
and the storage block determining unit is used for determining the storage block corresponding to the service data according to the theme of the service data.
11. The business data storage device of claim 9, wherein said device further comprises:
the consumption request acquisition module is used for receiving a consumption request of the service data;
the data storage determining module is used for determining a storage block in which the business data is stored according to the theme of the business data;
and the data sending module is used for sending the service data stored in the storage block so as to enable the service data to be consumed.
12. A server, comprising a processor and a memory, wherein the memory has stored therein at least one instruction, at least one program, set of codes, or set of instructions, which is loaded and executed by the processor to implement the business data storage method of any one of claims 1-6.
13. A computer readable storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by a processor to implement a business data storage method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810444053.0A CN108681578B (en) | 2018-05-10 | 2018-05-10 | Business data storage method, device, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810444053.0A CN108681578B (en) | 2018-05-10 | 2018-05-10 | Business data storage method, device, server and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108681578A CN108681578A (en) | 2018-10-19 |
| CN108681578B true CN108681578B (en) | 2021-07-16 |
Family
ID=63805837
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810444053.0A Active CN108681578B (en) | 2018-05-10 | 2018-05-10 | Business data storage method, device, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108681578B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10970309B2 (en) | 2019-06-05 | 2021-04-06 | Advanced New Technologies Co., Ltd. | Data storage method and apparatus |
| CN110347675A (en) * | 2019-06-05 | 2019-10-18 | 阿里巴巴集团控股有限公司 | A kind of date storage method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104156216A (en) * | 2014-08-14 | 2014-11-19 | 浪潮(北京)电子信息产业有限公司 | Heterogeneous storage management system and method oriented to cloud computing |
| CN105677805A (en) * | 2015-12-31 | 2016-06-15 | 北京奇艺世纪科技有限公司 | Data storing and reading method and device using protobuf |
| CN105765555A (en) * | 2013-11-06 | 2016-07-13 | 亚马逊科技公司 | Strict queue ordering in distributed system |
| CN107133309A (en) * | 2017-04-28 | 2017-09-05 | 东软集团股份有限公司 | Storage, querying method and the device of flow instance, storage medium and electronic equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6651108B2 (en) * | 1995-08-14 | 2003-11-18 | Next Software, Inc. | Method and apparatus for generating object-oriented world wide web pages |
-
2018
- 2018-05-10 CN CN201810444053.0A patent/CN108681578B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105765555A (en) * | 2013-11-06 | 2016-07-13 | 亚马逊科技公司 | Strict queue ordering in distributed system |
| CN104156216A (en) * | 2014-08-14 | 2014-11-19 | 浪潮(北京)电子信息产业有限公司 | Heterogeneous storage management system and method oriented to cloud computing |
| CN105677805A (en) * | 2015-12-31 | 2016-06-15 | 北京奇艺世纪科技有限公司 | Data storing and reading method and device using protobuf |
| CN107133309A (en) * | 2017-04-28 | 2017-09-05 | 东软集团股份有限公司 | Storage, querying method and the device of flow instance, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108681578A (en) | 2018-10-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102154446B1 (en) | Method for fast scheduling for resource balanced allocation on distributed and collaborative container platform environment | |
| EP3335119B1 (en) | Multi-priority service instance allocation within cloud computing platforms | |
| US8631403B2 (en) | Method and system for managing tasks by dynamically scaling centralized virtual center in virtual infrastructure | |
| Téllez et al. | A tabu search method for load balancing in fog computing | |
| CN105700948A (en) | Method and device for scheduling calculation task in cluster | |
| CN102082692A (en) | Method and equipment for migrating virtual machines based on network data flow direction, and cluster system | |
| CN102971724A (en) | Methods and apparatus related to management of unit-based virtual resources within a data center environment | |
| CN105187512A (en) | Method and system for load balancing of virtual machine clusters | |
| CN113641457A (en) | Container creation method, device, apparatus, medium, and program product | |
| JP2006085712A (en) | System and method for allocating computing resource for grid virtual system | |
| CN115328663A (en) | Method, device, equipment and storage medium for scheduling resources based on PaaS platform | |
| CN102567072A (en) | Resource allocation method, resource allocation device and resource allocation system | |
| CN108920153A (en) | A kind of Docker container dynamic dispatching method based on load estimation | |
| CN107864211B (en) | Cluster resource dispatching method and system | |
| CN105491150A (en) | Load balance processing method based on time sequence and system | |
| CN103885811B (en) | Method, system and device that dummy machine system total system is migrated online | |
| CN108681578B (en) | Business data storage method, device, server and storage medium | |
| WO2013082742A1 (en) | Resource scheduling method, device and system | |
| CN110162396A (en) | Method for recovering internal storage, device, system and storage medium | |
| CN114911598A (en) | Task scheduling method, device, equipment and storage medium | |
| CN113946431B (en) | Resource scheduling method, system, medium and computing device | |
| CN114598665A (en) | Resource scheduling method and device, computer readable storage medium and electronic equipment | |
| CN114389955A (en) | Embedded platform heterogeneous resource pooling management method | |
| CN106911741B (en) | Method for balancing virtual network management file downloading load and network management server | |
| CN106664259A (en) | Virtual network function capacity expansion method and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |